Evaluation of Rainfall Products in Semi-arid Areas: Application to the Southeast of the Republic of Djibouti and a Focus on the Ambouli Catchment

1. Introduction

The Horn of Africa is frequently hit by severe extreme events such as droughts and floods, but its climate remains poorly understood. The plains of Somalia and the Republic of Djibouti are arid, and despite their proximity to the equator, are under a low annual rainfall regime, 60-300mm [1]. Indeed, westerly winds lose most of their moisture after crossing the Sahel and Sudan [2] However, this region presents significant spatial and seasonal contrasts, partly due to its highly varied relief [3].

In the Republic of Djibouti, rainfall events are scarce. These events feed the flow of ephemeral rivers (wadi) and contribute to groundwater recharge but also trigger urban and surface flood hazards (the floods of 2004, 2018, 2019…). More generally, variability in rainfall rate, amount, and distribution substantially determines the Earth’s ecosystem, water cycle, and climate [4]. Therefore, accurate measurement of rainfall is essential to analyze the spatial and temporal patterns of precipitation at different scales and further to understand the impact of rainfall on agriculture, hydrology, and climate.

Conventionally, the ground rain gauge is the most widely used rainfall measurement instrument. This is the most accurate and reliable approach to point measurement of rainfall. However, rain gauge stations in many parts of the world and in most parts of Africa, including the Republic of Djibouti, are scarce and unequally distributed. Therefore, ground rainfall analysis is subject to high uncertainty when based on networks that have a low rain gauge density or are limited to a few isolated rain gauge stations. In this situation, the spatial representativeness of the measurement is closely related to the coverage of the rain gauges.

In this context, gridded precipitation datasets, hereinafter called P datasets, available on the Internet at different spatial/temporal scales offer a convenient alternative [5]. Such products have blossomed since the 1980s and represent a valuable opportunity because they are usually time continuous record, they provide a measurement in any portion of space, and they provide coverage with a daily or even hourly time step.

There are three basics P datasets, defined according to the input data and rainfall data retrieval process: those based on spatial information from rain gauges, those based on reanalysis data derived from physical and dynamic models, and those based on satellite information using passive microwave (PMW) and infrared (IR) information. It should be noted that a lot of P datasets combine the 3 types of data sources to obtain a better assessment.

In many regions of the world such as East Asia [6], South America [7] and West Africa [5,8], P datasets have already shown promising prospects for water resource management by improving the understanding of extreme events (droughts and floods) and of the spatiotemporal variability of precipitation. In East Africa, the use of P datasets has increased in recent years [9,10,11]. In catchments with a limited number of gauges, other studies have reported the need to corroborate the rain gauge measurements with products derived from satellite-based measurements, given their extensive and reliable global coverage [12,13]. Except for large-scale studies that consider the whole of East Africa, there have been very few reports on the spatiotemporal variability of rainfall in the Republic of Djibouti [1,14].

Assowe Dabar [1] provides a description of the spatial and temporal rainfall variability in Djibouti by analyzing monthly and annual rainfall data recorded at 14 stations. Firstly, a regionalization by a Principal Component Analysis is performed in terms of spatial distribution of rainfall, and two zones of seven stations each are identified, located in the west (inland region) and the east (coastal region) of Djibouti, respectively. Secondly, the performance of four gridded datasets of satellite rainfall estimates and of the ERA5 reanalysis data is evaluated on the base of the regionalization in these two zones and at Djibouti-Aerodrome, the longest meteorological record in Djibouti. They conclude that Satellite-based rainfall products with high resolutions and ERA5 reanalysis offer insightful information on the rainfall variations at the spatial scale of Djibouti country.

In semi-arid regions, the relationship between rainfall and runoff is of utmost importance due to the potentially destructive nature of flash floods in these areas. This is especially true in volcanic environments, where the bare surface volcanic soils can exacerbate the effects of flash floods. The scarcity of rainfall events in these regions also means that there are few streamflow events, making it essential to accurately identify and monitor the few events that do occur and provoke significant runoff.

In this study, we aimed to identify the most representative P datasets for hydrological modelling and their usefulness for a specified study area including the Ambouli catchment (794 km²) where pluviometric data are scarce. At the scale of this catchment, some hydrological data have been acquired in the past, but there are no concomitant pluviometric information which does not allow to implement an hydrological model at a daily time step. So, this evaluation will be based on quantitative aspects (quantitative metrics) of rainfall but also on qualitative aspects of rainfall (categorial metrics): detection of days with/without rain. This qualitative aspect of rainfall estimation products is not commonly studied but is important for hydrological modeling of small watersheds that react to rainfall events in a few hours or even days.

2. Study area and Datasets

2.1. Study area

The Republic of Djibouti is located in the Horn of Africa, at the junction of the Gulf of Aden and the Red Sea. The country covers an area of 23,200 km2 and lies between latitudes 10°9’N and 12°7’N and longitudes 41°8’E and 43°5’E. With a geology of volcanic origin, the Republic of Djibouti is characterized by a very steep relief featuring a succession of massifs, plateaus, and plains. The Republic of Djibouti has a semi-arid/arid climate mainly characterized by the irregularity and the weakness of precipitation, the absence of perennial streams, except in the mountainous regions, and a hot temperature year-round: “low” temperatures from October to April (20°C to 30°C) and “high” temperatures from May to September (30°C to 45°C). These climatic conditions are responsible for strong water evaporation of about 2,000 mm/year (potential evapotranspiration), which severely affects both pasture and the soil [15].

2.1.1. Southeast of the Republic of Djibouti

Assowe Dabar [1] regionalized the climate of the Republic of Djibouti on the basis of an analysis of precipitation and obtained two clusters, EAST and WEST (Figure 1a). The EAST zone concentrates most of the population and economic exchanges between Djibouti and its neighboring countries (Figure 1b). Therefore, our study focuses on the southeast part of the country, near Djibouti. Southeast area is about 5000 km². Its proximity to the Red Sea and Gulf of Aden drives most of the rainfall variability [1].

In the EAST zone, as in most of East Africa, it rains mainly in March-May (MAM) and October-December (OND). There are several hypotheses on the origin of these rains. During the transitional months (January and February), it is thought that these rains are associated with rare occurrences of tropical disturbances from the northwestern Indian Ocean, and in the winter, it is thought that they come from higher altitudes [1,16]

2.1.2. Catchment of the Ambouli Wadi

Inside the southeast region, we focus on the catchment of the Ambouli wadi, which is one of the country’s largest catchments (794 km²) and is of major importance (Figure 1c). The Ambouli aquifer is the main source of drinking water for the city of Djibouti.

Despite the importance of the catchment for the development of Djibouti City, it has been the subject of only a few scientific studies. Moreover, the increase in the population has been accompanied by a greater sprawl of Djibouti City [17]. Urbanization leads to greater sealing of the catchment, which modifies the quality of water seeping into the aquifer and affects aquifer storage, but above all increases the frequency and magnitude of floods. Seyoum [18] have shown these effects in various East African countries such as Kenya and Ethiopia. The frequency of extreme flood events in Djibouti City has been increasing over the last two decades. The events of the years 2004, 2013, and 2018 were responsible for much economic and human damage [14,19].

2.2. Datasets

2.2.1. Rainfall Estimate Products (P Datasets)

The P-dataset products were selected based on their availability in the two study areas and their relevance for operational research focusing on hydrology and/or agronomy [8]. To be relevant for the target application, products need a daily or sub-daily temporal resolution. Table 1 describes the 21 products. It should be noted, however, that the GPCC product has a monthly temporal resolution. Among the 21 products analyzed and compared to the rain gauge data, there are 15 long-term P datasets with more than 30 years of continuous observations and 5 P datasets spanning more than 15 years. For the southeast, the analysis is carried out on 15 long-term P datasets, and for the Ambouli catchment, 11 P datasets (Table 1).

2.2.2. Rain gauges of the National Meteorological Agency

At this scale, we will be working at monthly to annual time steps, except for Djibouti-Aerodrome where we will also be working at the daily time step. There are 14 rain gauge stations located in the southeast of the Republic of Djibouti managed by the Agence Nationale de la Météorologie de Djibouti (ANM). However, the main weakness of the rain data available from this network is the gaps in the time series. Based on the criteria of having more than 10 months of data per year, a total of 5 rain gauges were selected for analysis, namely Djibouti-Serpent, Djibouti-Aerodrome, Ali-Sabieh, Hol-Hol, and Loyada (Figure 1b). These are tipping-bucket rain gauge. For the comparison with the 15 P datasets, the common period of analysis will be 1980-1990. At the daily time step only the Djibouti Aerodrome rain gauge is available for the southeast for the period of analysis (1981-2010).

2.2.3. Rain gauges of the Early Warning System

Following the catastrophic floods in Djibouti City in 1994 and 2004, the government of the Republic of Djibouti has set up protective measures. These measures included the installation of the USAID-funded Wadi Ambouli Flood Monitoring and Alert System (RACA). This system was installed in the Ambouli catchment between February and March 2008. The hydrogeology and hydrology laboratory of the CERD (Centre d’Etudes et de Recherches de Djibouti) managed this network of 7 rain gauge stations (Figure 1c) until 2015. These are tipping bucket rain gauges. For some of them, rainfall is measured at a sub-daily time step with no long periods of missing data (less than ten days per month and less than four months per year missing). For the comparison with the 11 P datasets, the common period is 2008-2013

2.2.4. Selected rain gauges and common periods of study

The two study areas mentioned above, the southeast of the Republic of Djibouti and the Ambouli catchment, are covered by respectively the ANM and the RACA network. The datasets are available for different periods and are analyzed at different time steps (Table 2 and Figure 1).

3. Methodology

3.1. Pre-processing (pre-treatment)

Since the selected P datasets differ in spatial resolution, from 0.0375° for TAMSAT v.3 to 1° for GPCC v.7, all the datasets used in this study were resampled to a spatial resolution of 0.1° by 0.1° for easy comparison. The P datasets with a spatial resolution > 0.1° (PERSIANN-CDR, ERA-5, MERRA-2…) were downscaled to 0.1° using nearest neighbor, while the dataset with a spatial resolution < 0.1° (CHIRPS, TAMSAT…) was uscaled to 0.1° using bilinear interpolation[5,37] Moreover, some of the datasets are at the daily time step, while others are at the sub-daily time step. In order to obtain homogenous data, we aggregated the sub-daily data at the daily time step over the study period.

3.2. Approach

This study assesses the performance of 21 P datasets at different temporal resolutions including daily, monthly, seasonal, and annual time scales as summarized the Figure 2. The evaluation is conducted on two distinct spatial scales and over two time periods: first, the southeast zone of the Republic of Djibouti using 15 long-term P datasets for the common period of 1980-1990, and the Ambouli catchment area using 11 P datasets for the period of 2008-2013. The prerequisite for the choice of comparison period was that the period be common to all the P datasets and all the rain gauge data. It is worth noting that the 5 P datasets used in both analysis areas bring the total number of P datasets to 21. The choice of P datasets for the southeast zone was based on their availability over the selected period, while the selection for the Ambouli catchment was based on their spatial resolution, which needed to be less than or equal to 0.1.

The validation of the P datasets was carried out on two spatially aggregated levels: point-to-pixel and catchment scale. The point-to-pixel scale is a pairwise comparison of the rain gauge data from the ground station with the corresponding gridded precipitation value.

The catchment-scale validation analysis was performed by comparing spatially aggregated P datasets for the catchment to a corresponding interpolated rainfall based on the ground reference rainfall. The catchment spatial unit is particularly relevant for characterizing rainfall because it represents the conventional working unit for hydrological applications. At the same time, and as suggested by Xie and Arkin [38], aggregation on a larger scale enhances the stability of the precipitation validation exercise. For the interpolation of the mean annual rainfall at the catchment scale, we used the Inverse Distance Weighting [39] spatial interpolation method. This is a local deterministic interpolation technique that calculates the value of a point by averaging the values of the neighboring points weighted by the inverse of the distance to the calculated point: the closer the points, the greater the weight assigned.

3.3. Metric validation

To assess the ability of P datasets to estimate the amount of ground reference rainfall, the metric analysis includes detailed quantitative and categorical metrics. Although a wide range of evaluation metrics are available to assess data performance [4,40,41,42], there is no single metric that encapsulates all the aspects of interest. For this reason, it is useful to consider several performance metrics and understand the type of information or insight they might provide [43].

3.3.1. Quantitative metrics

Three quantitative metrics were used for all the time steps (Table 3):

• The Root Mean Square Error (RMSE) is a frequently used measurement of the difference between two variables. It measures the average magnitude of the estimation errors: low RMSE values indicate a low variance in the P datasets. To calculate the RMSE, we used the median value of a P dataset.
• The Mean Bias Error (MBE) is used to estimate the average bias in the P datasets and provides a good indication of the mean under- or overestimation of the predictions. A positive value of MBE means an overestimation. To calculate the MBE of a P dataset, we used the median value.
• There are several indices used to evaluate the performance and reliability of P datasets over time. We used the Kling-Gupta Efficiency (KGE) because water resource management requires reliable representation of precipitation temporal dynamics (measured by r) and volume (measured by $\beta$ and $\gamma$) [5]. However, for this study we will only analyze the value of the KGE in its totality. Thus, the median value of the KGE is used as a reliability criterion at the scale of the study area.

In Table 3, x corresponds to the ground reference rainfall and y corresponds to the rainfall of the P datasets.

$$r=\frac{1}{n}\sum _{i=1}^n\frac{\left(xi-\overline{x}\right)\times \left(yi-\overline{y}\right)}{\left(\sigma x\times \sigma y\right)}$$

(1)

$r$ is the Pearson correlation coefficient and is used to evaluate the goodness of fit of the P-dataset estimations to the observed data.

$$\beta =\frac{\overline{y}}{\overline{x}}$$

(2)

$\beta$ is the ratio between the averages of the P datasets and of the observed data.

$$\gamma =\frac{1}{\beta }\times \frac{\sigma y}{\sigma x}$$

(3)

$\gamma$ is the ratio between the coefficients of variation of the P datasets and of the observed data.

3.3.2. Categorical metrics

The performance of the P datasets was also analyzed using categorical metrics at the daily time step [5]. Five statistical indicators based on a contingency table (Table 4) were computed to constitute the categorical metrics (Table 5):

• The probability of detection (POD) indicates what fraction of the observed events was correctly estimated.
• The false alarm ratio (FAR) corresponds to the proportion of events identified by the P datasets but not confirmed by gauge observations.
• The accuracy measures the proportion of correct decisions.
• The error measures the proportion of incorrect decisions.
• The Heidke skill score (HSS) evaluates the ability of the P datasets to detect precipitation events in comparison to a random prediction. The HSS values range from -∞ to 1, with a perfect score of 1 and negative values indicating that random prediction outperforms the P dataset.

The HSS value was computed at each grid-cell location to observe P-dataset reliability over space.

4. Results

The performance of the P datasets was evaluated using the quantitative metrics (i.e. KGE, RMSE and MBE) and categorical metrics (i.e. POD, FAR, Accuracy, Error, and HSS) at the annual, seasonal, monthly, and daily temporal scales.

4.1. Annual comparison

4.1.1. Southeast of the Republic of Djibouti (1980-1990 / 15 P datasets)

The KGE values reveal strong agreement with the rain gauge data for some of the P datasets (Figure 3). Eight of the P datasets (CPC, ERA5, EWEMBI, GPCC, JRA-55 Adj, MERRA-2 PT, MERRA-2 PTC and WFDEI-GPCC) overestimated rainfall (positive MBE). Five P datasets have high KGE values: EWEMBI, GPCC, JRA_55 Adj, MSWEP v.2.2 and WFDEI-GPCC. These five P datasets also have the lowest RMSE values (between 75 and 100 mm per year). Moreover, WFDEI-GPCC is the best among all the P datasets (using the KGE performance criterion), with the most satisfactory values of MBE and RMSE, while MERRA-2 PTC and ERA 5 have the worst performance. It is worth mentioning that the Loyada rainfall station has the lowest values of KGE except for TAMSAT and WFDEI-CRU (Figure 3).

4.1.2. Ambouli Catchment (2008-2013 / 11 P datasets)

The Figure 4 shows the interpolated average annual precipitation of the 11 P datasets compared to the average annual precipitation of the rain gauge stations. We first notice that the general west-east rainfall gradient, which is characterized by a decrease in mean annual rainfall, was captured by most of the analyzed products. IMERG-LR presents the highest annual average (308 mm year^-1, with a standard deviation of 25 mm year^-1) and GSMaP-Adj the lowest (79 mm year^-1, with a standard deviation of 17 mm year^-1), which corresponds to a wide range of annual rainfall from 79 to 308 mm. The average annual rainfall according to the rain gauge stations is 90 mm year^-1. We have thus identified the 4 P datasets out of the 11 that present the lowest standard deviation in terms of annual mean (ARC2, CHIRPS, GSMaP-Adj and MSWEP v.2.2), with MSWEP v.2.2 the most reliable product compared to the rain gauge stations (with a slight overestimation of the annual rainfall values by 10%). MSWEP v.2.2 product remains effective at both scales. The least performing products are the IMERG products (ER, FR and LR).

4.2. Seasonal comparison

4.2.1. Southeast of the Republic of Djibouti (1980-1990 / 15 P datasets)

The climate of the country is characterized by two distinct periods:

• a wet period from October to April that is characterized by low but friendly temperatures (between 22 C and 30°C) and a relatively high humidity
• a dry period that results in high temperatures (between 30°C and 40°C) and a sandy wind, dry and hot (locally called the Khamsin) which runs from May to September [44].

Figure 5 presents the metric validation at the seasonal scale. We observe that 80% of the P datasets perform better during the wet season than during the dry season. Like for the annual time step, EWEMBI, GPCC, JRA-55 Adj, MSWEP v.2.2 and WFDEI-GPCC are the best-performing P datasets.

The KGE values of these five P datasets vary considerably, around 0.5 for the wet season and much lower for the dry season. For all the P datasets, the RMSE values for the wet season are on average more than three times higher than for the dry season. The MBE values show that the five P datasets generally overestimate the rainfall (positive MBE values), but we do observe two underestimated values, corresponding to MSWEP v.2.2 (wet season) and WFDEI_GPCC (dry season).

WFDEI-GPCC is the most reliable product at the seasonal scale (wet and dry periods). MERRA-2_PTC is the least accurate product, with high RMSE and MBE values and a KGE value lower than 0.2 (Figure 5).

4.2.2. Ambouli Catchment (2008-2013 / 11 P datasets)

Just like in the seasonal analysis for the southeast of the country, in the Ambouli catchment:

• The P datasets mainly overestimate the seasonal rainfall but perform better in the wet season than in the dry season.
• CHIRP and CHIRPS perform better in the dry season but have a low KGE value around 0.2.

However, contrary to the above seasonal analysis, the RMSE values for the dry season and for the wet season have an almost identical mean, which means that for this period, we have a rainy season spread over both the wet and dry periods.

We also notice that the best performing products at the seasonal time step are GSMAP-RT and MSWEP v.2.2, which are the same as for the annual time step.

4.3. Monthly comparison

4.3.1. Southeast of the Republic of Djibouti (1980-1990 / 15 P datasets)

The metrics are listed in Table 6. Strong agreement with the rain-gauge data is observed for some of the P datasets. At the monthly time step, we observe results similar to those obtained at the previous time step (Figure 3):

• EWEMBI, GPCC, JRA-55_Adj, MSWEP v.2.2 and WFDEI_GPCC, the five most reliable P datasets at the previous time step, show the best statistical scores at the monthly time step (Figure 3).
• WFDEI_GPCC has the best performance (KGE=0.70, RMSE = 19.61 mm month^-1 and MBE= 1.27 mm month^-1).
• MERRA-2 PTC is the least reliable product.
• Loyada is the rain gauge station that presents the least satisfactory KGE values.

4.3.2. Ambouli Catchment (2008-2013 / 11 P datasets)

Table 7 shows the values obtained for the quantitative metrics at the monthly and daily time steps. MSWEP v.2.2 have the highest KGE (0.32), the lowest RMSE (15.5 mm.month^-1) and the lowest MBE (near to 0 mm month^-1). This product also has the best performance at the annual and seasonal time steps. On the contrary, the IMERG products (ER, FR and LR) show the least satisfactory results, with a negative KGE value and the highest RMSE and MBE values, like for the previous time step. At the monthly time step, the KGE values obtained for the southeast of the country are higher than those obtained for the Ambouli catchment (0.70 versus 0.35 for the most reliable P datasets).

Only MSWEP v.2.2 remains effective for both spatial resolutions and temporal windows, as well as for the previous time step.

4.4. Daily comparison

4.4.1. Southeast of the Republic of Djibouti: Djibouti-Aerodrome rain gauge (1981-2010 / 15 P datasets)

For the daily time step analyses, we use data from the only Djibouti-Aerodrome rain gauge for the period 1981-2010, which is the common period between the station and all the P datasets. The GPCC product is not present in the analysis of the daily time step data because it has a monthly temporal resolution.

The ability of the P datasets to quantify the amount of daily precipitation is relatively low, with most products having negative KGE values (Table 6). Only the most reliable products at the previous time step, EWEMBI, JRA-55_Adj, MSWEP v.2.2, and WFDEI_GPCC, have positive KGE values. With regard to MBE, the values are relatively homogeneous for all the products (from -0.13 mm to 0.42 mm). The low values of MBE and RMSE are due to the low number of rainy days (i.e. with a rainfall of > 1 mm day^-1).

Table 8 shows the results of the POD, FAR, Accuracy, Error and HSS metrics. Some values such as POD, Accuracy and Error give "relatively good" values but are in fact largely skewed due to the very few rain events observed. For the POD, the P datasets MERRA-2 (PT and PTC) and ERA5 (the worst performing products at the previous time steps) detect almost no rainy days, which explains the almost good POD values for these products. EWEMBI, JRA-55 Adj, and WFDEI-GPCC provide a satisfactory POD of > 0.7. For the FAR, we observe relatively high values for the products (FAR > 0.9 except for CHIRPS, which presents a FAR of 0.75). For the Accuracy and Error, the results are moderately satisfactory for most of the products; however, like for the FAR values, the CHIRPS product has the best Accuracy and Error. Indeed, compared to rainfall data, these products detect events rather well when they occur (POD and accuracy) but they detect much more events than in reality (FAR and Error). Compared to the other products, MSWEP has however a poor POD.

The HSS values for all the products are very low (almost zero). The highest HSS score was obtained by CHIRPS (HSS = 0.19). Given the number of non-rainy days, it seems necessary to set a threshold value (rainfall threshold). Thus, several threshold values ranging from 1 mm to 50 mm were applied to the products for the calculation of HSS [5], and the maximum HSS was obtained with a threshold value of 1 mm. After applying the threshold value of 1 mm, we observe that the HSS values are higher, this probably shows that there are more detected rainfall < 1mm and the products with the best performance are the same as at the annual and monthly time steps (EWEMBI, JRA-55 Adj, MSWEP v.2.2 and WFDEI-GPCC), as well as CHIRPS. Therefore, the optimum of the "threshold HSS" is HSS > 1 mm, which probably shows that there is limited concomitance between the P-datasets and the observed data for the extreme rainfall events (this analysis is more detailed in the next paragraph).

With an average annual rainfall of 150 mm at the Djibouti Aerodrome station, the average number of rainy days is 12 days (higher than 1 mm). However, the most efficient products at daily time step, MSWEP, CHIRPS, and JRA-55 give an average of 16, 21, and 24 rainy days respectively (for rainfall higher than 1 mm). In contrast, the other products provide results with an average of more than 40 days (with a maximum of rainy days for the MERRA-2 product with 86 rainy days on average).

To complement this analysis, we also investigated the ability of these P-datasets to detect rainfall events triggering runoff. For this purpose, a threshold of rainfall greater than 10 mm (which corresponds to an average of 3 events per year) was used. The percentage of concomitant rainfall days was calculated for each P dataset, with values ranging from 44% for MSWEP to 80% for EWEMBI for the efficient products.

After obtaining the concomitant day values, we analyzed the concomitance of the rainfall values and observed that, on average, 95% of rainfall events higher than 10 mm are significantly underestimated by the P datasets. There was no concomitance observed between the P datasets and extreme rainfall values (greater than 50 mm). These results highlight the limitations of these P-datasets for rainfall-runoff modeling at a daily time step and underscore the need for further research to develop more accurate and reliable rainfall datasets for high flood forecasting in semi-arid regions.

4.4.2. Ambouli Catchment (2008-2013 / 11 P datasets)

Similarly to the daily analysis in the southeast, the ability of the P datasets to quantify the amount of daily precipitation is relatively low. In terms of the KGE values (Table 7), only GSMaP-RT and MSWEP v.2.2 (the best-performing P datasets at the previous time steps) present values greater than 0.3. These values are homogenous with those obtained at the previous time step. The RMSE and MBE values of these two products are satisfactory, with an MBE close to 0 mm and an RMSE close to 2 mm.

To complement the analysis of the quantitative metrics, we analyzed the selected categorical metrics (Table 9):

• The POD and FAR values are mainly high because a mean of 11 rainy days was recorded per year at the Ambouli catchment (these results are in agreement with the results obtained in the southeast of Djibouti), and rainfall estimation products in general overestimate the number of rainfall event in an arid environment, which results in FAR values close to 1 (optimum 0).
• The Accuracy and Error values for the P datasets (other than the CHIRP product) are respectively on average close to 0.9 and 0.1. These values are satisfactory.
• It is the HSS that allows us to quantify the reliability of the P datasets in reproducing daily rainfall. We note that the values of HSS without a threshold value are almost zero, while the values of HSS with a threshold at 1 mm are higher (for some products values close to 0.3), which agrees with the study conducted by Satge [5] and the point-to-pixel analysis (southeast of the Republic of Djibouti). The highest scores in terms of HSS, with a threshold of > 1 mm, were obtained by GSMaP-RT, MSWEP v.2.2 and CHIRPS. Therefore, the results obtained in the Ambouli catchment, are similar to those obtained in the southeast zone.

5. Discussions

In recent years, the use of P datasets has been increasing in areas with a low rainfall network density due to the ease of access to data and the spatial coverage of these products.

In this study, some P datasets provide good performance at all time steps. For the southeast of the Republic of Djibouti, three out of five of the best-performing products (EWEMBI, JRA-55 Adj and WFDEI-GPCC) are reanalysis products that are merged with gauge-based information. For the Ambouli catchment, the best-performing P datasets are satellite-based products that are merged with gauge-based information (except for GSMaP-RT, which is a pure satellite product). Thus, the best-performing products are those that integrate rain gauge information into their algorithm, which confirms the previous results obtained in Africa [8,45,46] and other regions of the world [37,47,48].

For this study, CHIRPS performs better than CHIRP at all time steps. However, GSMaP-RT outperformed its adjusted version, GSMaP-Adj, at the monthly and daily time steps. This result is consistent with previous observations over the CONUS (CONtiguous US), [5,37] and illustrates the potential limit of P-dataset algorithms to consider the best gauge data. GSMaP-RT v.6 is the only P dataset with no gauge-based information that ranks in the top two monthly and daily at the catchment scale. Therefore, GSMaP-Adj v.6 should be highly effective if it used gauge-based information in the optimal form [5].

Only MSWEP v.2.2 P is reliable at all time steps and for both time periods considered. The strong performance of MSWEP v.2.2 might be due to the integration of daily gauge observations for the calibration of rainfall estimates [11,32,37]. In comparison to the rain gauge of Djibouti-Aerodrome, this product underestimates heavy rainfall and overestimates light rainfall, but still manages to detect the rainy days. CHIRPS is among the best products at the daily time step over the two periods of the study. These results agree with several comparative studies from the East African region [49]. However, for most of the P datasets, there is a degradation in the performance from the annual to the lower time steps.

It is advisable to be attentive to the P-dataset version. Our results show that MSWEP v2.2 has a better performance in terms of rainfall data accuracy. This best identified product MSWEP v2.2 has a newer version, MSWEP v2.8. One could have expected performance increasing with version. But, when analyzing the rainfall distribution, MSWEP v2.2 shows a better fit to the observed data than MSWEP v2.8 [50]. In addition, we find that MSWEP v2.2 provides more consistent data for semi-arid areas exhibiting low rainfall, in contrast to MSWEP v2.8 which shows inconsistencies in these areas. MSWEP v2.2 outperforms the new MSWEP v2.8 version for estimating rainfall data in semi-arid regions. These results highlight the importance of continuous evaluation of rainfall data models to assess their performance and accuracy.

According to Satge [5] and Omonge [11], products dedicated to the African continent (TAMSAT and ARC-2) do not always perform well. For the southeast of the Republic of Djibouti, our results confirm this low performance (KGE and HSS) over the years 1980-1990. However, ARC2 is in the top three over the period 2008-2013 at the catchment scale at the annual and monthly time steps and gives satisfactory results at the daily time step (ranked 4/11 in terms of performance). Therefore, as for most P datasets, the use of a new rainfall estimation algorithm may be advantageous, but it may not be the case depending on each model. It would be recommendable to select a set of fairly well performing models (and not only just the best one), and test the performance of the last two versions of the same models, in order to span the possible optimal point of representativeness of rainfall products. This method should be continued in the future on the region for assessing the progress of models in representing the rainfall variability.

This study focused on long-term P datasets, but also on more recent P datasets (created in the 2000s) at the catchment scale. More recent products perform better than those incorporating historical data into their database (except for MSWEP v.2.2), which is probably due to the fact that the most recent products integrate new innovations (high-definition satellite images, new passive microwave sensors) into their algorithm.

6. Conclusion

Our objective was to determine the best performing rainfall estimation products in the study area for use in supplementing the limited data available for use in hydrologic modeling. To achieve this objective, we compared the performance of different rainfall estimation products using two types of metrics: quantitative (KGE, RMSE and MBE) and categorical metrics (POD, FAR, Accuracy and error). The quantitative metrics measured the accuracy of rainfall estimation in terms of numerical values. The categorical metrics, on the other hand, were used to evaluate the ability of the rainfall estimation products to correctly detect rainfall events. The use of categorical metrics was particularly important in our study, as it allowed us to better evaluate the ability of the different rainfall estimation products to reproduce the temporal variability of rainfall. We compare P datasets to rain-gauge data at two spatial resolutions (southeast of the Republic of Djibouti and the Ambouli catchment) over two temporal windows (1980-1990 / 2008-2013).

For the southeast of the Republic of Djibouti, at the point-to-pixel scale, five P datasets present high KGE values at the annual and monthly time steps: EWEMBI, GPCC, JRA-55 Adj, MSWEP v.2.2 and WFDEI-GPCC. At the seasonal time step, all the P datasets reproduce the seasonality relatively well but overestimate the precipitation amounts. At the daily time step, the five best products in terms of KGE and HSS are CHIRPS, EWEMBI, JRA-55 Adj, MSWEP V.2.2 and WFDEI-GPCC. The least effective products are the MERRA-2 products (PT and PTC) and the IMERG products (ER, FRand LR).

At the scale of the Ambouli catchment, the best products are: At annual and seasonal time scales, in terms of average annual rainfall over the catchment: MSWEP v.2.2, GSMaP-Adj, ARC2, and GSMaP-RT. At monthly time scale, in terms of KGE value: MSWEP v.2.2, GSMaP-RT, ARC2 and GSMaP-Adj. At daily time scale, in terms of the metrics used (KGE and HSS): MSWEP v.2.2, GSMaP-RT, and CHIRPS.

Despite a sparse rainfall measurement network and often discontinuous data, some observations emerge from the analyses. The performance of the P datasets degrades from the annual time scale to the daily time scale. The performance of most of the products also differs from one spatial scale to another. Our results demonstrate the relevance of considering different temporal and spatial windows and illustrates that a given P dataset performs differently according to the spatiotemporal scales.

Strong discrepancies between rain gauge observations and P datasets are illustrated. However, MSWEP v.2.2, GSMaP-RT and CHIRPS products can be used to indicate an order of magnitude of rainfall in the southeast area of Ambouli, especially if we consider long inter-annual series. MSWEP v2.2 outperforms the new MSWEP v2.8 version for estimating rainfall data in semi-arid regions. These results highlight the importance of continuous evaluation of rainfall data models to assess their performance and accuracy.

Nevertheless, even the most effective products are not suitable for flood forecasting using rainfall-runoff models at the daily time step. This is supported by our analysis of categorical metrics and the concomitant rainfall days. While some products, such as CHIRPS and MSWEP v2.2, showed better performance in terms of overall rainfall detection and days of concomitant rainfall, they still fell short in accurately predicting extreme rainfall triggering floods.

While the performance of some of the P-datasets is acceptable at the monthly and annual time steps, the poor performance at the daily time step is a major limitation for our study. This finding highlights the need for developing more accurate and reliable P-datasets that can capture the spatiotemporal variability of daily rainfall in the region and especially the highest events. In addition, further research is needed to investigate the factors that influence the performance of P-datasets in the region, such as the impact of topography, land cover and land use.

Overall, our study provides important insights into the limitations of P-datasets for flood forecasting in the Ambouli catchment and underscores the need for continued efforts to improve the accuracy of rainfall data for hydrological applications. In conclusion, while P-datasets are useful tools for studying precipitation patterns and trends, they should be used with caution for flood forecasting in semi-arid regions. Local data sources and more sophisticated modeling approaches are necessary for accurate and reliable flood prediction in these vulnerable regions.