2.3. Performance Evaluation of CMIP6GCMs
Performance evaluations of the CMIP6 models in this study were undertaken after extracting the daily data of the five CMIP6 GCMs (
Table 3) for the 20 meteorological stations in the study area (Appendix Table 1) for the period 1995-2014. The station (point) data and each CMIP6 GCMs data were the areal averages for the Western Amhara scale, which were evaluated by comparing historical CMIP6 GCMs against observed rainfall data at daily, monthly ,and seasonal timescales separately. The performance of CMIP6 models under different Shared Socio-economic Pathways (SSPs) were evaluated in simulating future rainfall characteristics in the Western Amhara region based on efficiency criteria found under “hydroGOF” R package (Gashaw
et al. 2024). Numerous performance efficiency criteria exist for model evaluation (Gashaw
et al. 2024).However, most authors prefer to use performance efficiency criteria, including the root-mean square errors (RMSE), percentage of bias (PBIAS),and Pearson correlation coefficient (r) (Alaminie
et al.,2021; Gashaw
et al. 2024). The RMSE measures the average magnitude of the deviation of rainfall products from the ground based observed data (Equation 1). A smaller RMSE value indicated that rainfall products were closer to the observed rainfall data. The optimum value of RMSE is 0 (Ageet
et al. 2022). The PBIAS measures the average tendency of rainfall products data to be larger or smaller than the observed data. Percent bias (PBIAS) is a statistical metric that spans from -∞ to ∞, with an ideal value of zero. The low magnitude of the PBIAS suggests that the rainfall products accurately estimate the observed data. Negative values of PBIAS indicate a bias of overestimation in the rainfall products, whereas, positive values of PBIAS indicate a bias of underestimation in the rainfall products when compared to the observed data (Gupta
et al.1999). Rainfall products are said to be a reliable rainfall measurement sources where PBIAS value ranging between -10% and 10% (Dangol
et al. 2022). PBIAS can be calculated using (Equation 2). The correlation coefficient (r) was used to measure the goodness of fit and linear association between variables, in this case the linear association between the observed rainfall and rainfall product data (Equation 3). The values of r range from -1 to 1, with r = 0 indicating no linear relationship, and r = -1 or 1 indicating a perfect negative or positive score (Dinku
et al. 2018; Ageet
et al. 2022;Aniley
et al. 2023).
observed rainfall total at a gauge station,
is the mean of observed rainfall total at gauging station, S is rainfall total for CMIP6 GCMs rainfall product,
is the mean of CMIP6 GCMs rainfall total, N is the number of data pairs compared and r is Pearson’s correlation coefficient. The overall ranking of the five GCMs considering all statistical metrics (i.e., r, RMSE, and PBIAS) were undertaken following the Comprehensive Rating Index (CRI) method using Equation (Equation 4).
Where n is the number of statistical performance measures used in this study for evaluating the models (5), m is the number CMIP6 GCMs that are evaluated in the study (
Table 1), and Ranki is the rank of the CMIP6 GCMs for each performance measures, which ranges from 1 to 5 for best and low performing models, respectively. The closer the values of CRI to 1 indicates the better performance of the model (Gashaw
et al. 2024).
Bias corrections of the global dataset were performed using Distribution Mapping (DM) technique. Distribution Mapping (DM),which is available in the Climate Model data for hydrologic modeling (CMhyd) tool (Gashaw et al. 2024),is used for bias correction of best performing GCMs projections under different climate change scenarios. The selection of DM for bias correction of GCMs for rainfall was based on the suggestion of Worku et al.(2020), who compared several bias-correction techniques available in CMhyd in the Jemma sub-basin, Upper Blue Nile Basin of Ethiopia and suggested the application of DM for bias correction of rainfall and temperature products. Another study in the Awash Basin of Ethiopia also found that DM is suitable for the bias correction of climate models for temperature (Tadese et al. 2020). This study bias corrected the best performing CMIP6 GCMs (top three ranked) after extracting the data for each stations, hence bias correction was performed by providing the daily observed historical (1995–2014), raw GCMs historical (1995–2014) and future raw GCMs (2021–2100) data of each station in text file. In this study, the analysis of future climate for the near future (2021–2050), mid-century (2051–2080) and late-century (2081–2100) periods (Belazreg et al. 2023) was conducted at SSP2-4.5, SSP3-7.0 and SSP5-8.5 climate change scenarios for the Western Amhara level. Scenarios SSP2-4.5, SSP3-7.0 and SSP5- 8.5 scenarios are representing medium-forcing, medium to high forcing and high-end-forcing pathways, respectively. The main reason why this study performed the future climate change analysis in these scenarios is that three of them are possible climate change trajectories. On the other hand, the one that is not considered in this study (i.e., SSP1-2.6, representing the low climate forcing pathway)is unlikely to occur considering the current global actions undertaken to mitigate climate change based on the Paris Agreement. Therfore, future climate change analysis and bias corrections of global dataset were performed mainly for SSP2-4.5, SSP3-7.0 , and SSP5-8.5 climate change scenarios (Gashaw et al. 2024).
2.3.2. Determination of the Past Rainfall Characteristics
The quality-controlled rainfall data were organized used in excel 2010 spread sheet following the days of the year (DOY) entry format, and the daily rainfall data was subjected for detailed analysis using sequences of statistical packages. The appropriate OD, CD, and LGP was calculated and simulated by using the CDT tool (Dinku
et al. 2022). The onset date was calculated based on the following criteria , the day that recorded 20mm of rainfall over five consecutive days without being followed by a dry spell longer than seven days within 21 days of planting day and with at least three rainy days on which threshold for rainy days (> =0.85mm) (Dinku
et al. 2022).This was done to capture both the early start and the late start, and the condition that there should be no 7-day dry period is to ensure that there is no false start (Dinku
et al., 2022). The end of the growing season is the first occasion when the water balance drops to zero after the end of rain (Zeleke
et al. 2023).In this study the end of the rainy season was calculated using the CDT tool with the criterion that, the water balance drops below 5mm for a period of three days (Dinku
et al. 2022).The length of the growing season (LGP) for the main rainy season was determined as the number of days between the rainfall onset and cessation dates using Equation (5).
Trend analysis: After investigating /determining the OD, CD, LGP of the kiremt season using CDT tool, for the study area and for study time period (past and future) the timeseries pattern was analyzed by using Sen’s slope estimator and modified Mann-Kendal test (non-parametric trend test) using R-statistical Software at 5% significance level. The Mk trend test is the most suitable and favorable non-parametric test for identifying trends in timeseries climate data. This method is less influenced by missing values and uneven distribution and is less sensitive to outliers, because it considers the ranks of observations rather than their actual numbers (Belay et al. 2019).
The S-statistic was applied to check for an increasing, decreasing or no-change trend in the OD, CD, and LGP data series in each of the selected meteorological station data in the MK t- test statistic, which is given as follows:
The values Xj and Xk in Equation (6) are the measurements corresponding to times j and k, and n is the number of events. To estimate the variance of S: -
In Equation (7), n: number of observations; m: number of data pairs in the series; ti : number of ti’s at the timestep i (Silva 2017; Tadese
et al. 2019). Hamed and Ramachandra Rao (Hamed & Ramachandra Rao 1998) proposed a modified version of the MK test (MMK), which is a better alternative than the original MK test for detecting trends in time series exhibiting autocorrelation, a common characteristic in hydrological time series. The difference is in the formula used to calculate the variance for the MMK test:
where Var(S)* is the modified variance. With r R k corresponding to the lag-k autocorrelation coefficient of the ranks of the data, the correction factor n/n* is:
To determine the significance of the trend, both MMK and MK follow what is presented in Equation (10):
Trend analysis employs the Theil–Sen slope estimator (SS), a non-parametric metric that approximates the magnitude (Arrieta-Castro
et al. 2020). This metric is determined by the slopes of the lines (Qi) connecting the data pairs (N), with their values obtained through the following Equation:
Finally, the arrangement of the Q values in ascending order allows for the computation of the median using the Equation:
The trend of OD, CD, LGP, and kiremt season rainfall total for all sample stations and study time periods were easily calculated using R statistical package.
Variability analysis: The coefficient of variation (CV) is a relative measure of variability, that displays standard deviation relative to its mean (Morales-Acuña
et al. 2021). As a result, the coefficient of variation (CV), was used to determine the variability of the Western Amhara region OD, CD, LGP, and seasonal rainfall total for the past (1991-2020) using Equation (13).
Where: CV (%) is the coefficient of variation of OD,CD, LGP, and kiremt season rainfall for sample station, S is the standard deviation of OD, CD, LGP, and kiremt season for stations,µ is the mean of OD,CD, LGP, and kiremt season for stations during 1991-2020 time periods. The CV (%) can be interpreted as follows (Morales-Acuña et al. 2021).
When 0≤CV(%) <20, there is low variability between the given data of the given time scale.
When 20≤CV(%) <30, moderate variability was observed between the given data for the given time scale.
When CV(%) ≥30, there is high variability between the given data of the given time scale. Because of this, the magnitude of CV (%) of selected stations and regional scale were compared.
The Standardized Anomaly Index ( SAI) is a measure of variability, used to detect the variability and nature of the trend (Asfaw
et al. 2018). It was determined using Equation (14).
Where, Z is number of standard deviations of the observation deviated from the normal, x is an observed rainfall value and is mean rainfall and SD is the standard deviation. This statistic enables us to determine the dry (negative values) and wet (positive values) years in the observation. The rainfall anomaly value less than -1.65 is considered extremely drought, whereas a rainfall anomaly index greater than 1.65 considered to be extremely wet. The total Kiremt rainfall for the past (1991-2020) were calculated and identified as wet and dry years.
The PCI is used to examine the variability (heterogeneity pattern) of rainfall at different scales (annual or seasonal). The PCI value can help to identify periods of high rainfall that may lead to flooding as well as dry spells that can contribute to drought conditions. The PCI values were computed, as described by Oliver (1980) and modified by De Luis
et al. (2011). PCI can be calculated using Equation (15).
where: PCI is the precipitation concentration of the Kiremt season, and Pi is the rainfall amount of the ith month. According to Oliver (1980), PCI values less than 10 indicate a uniform monthly distribution of rainfall (low precipitation concentration), values between 11 and 15 denote moderate concentrations, values from 16 to 20 indicate high concentration, and values of 21 and above indicate very high concentrations (Asfaw et al. 2018).
Analysis of rainfall total: In a Box and whisker plots, the box represents the middle 50% of the whole data set, while whiskers represent the magnitude of the spread of the rest of the data set about the median or mean (Stern et al. 2006).The minimum, maximum, and mean daily rainfall total were summarized using a Box and Whiskers plot using the CDT tool (Dinku et al. 2022). The monthly and seasonal rainfall total were spatialy interpolated in ArcGIS software using the inverse distance weight interpolation technique for the past (1991-2020).