Generalization Ability of Bagging and Boosting Type Deep Learning Models in Evapotranspiration Estimation

1. Introduction

Crop water requirements form the major basis for irrigation water management. Evapotranspiration (ET), a word used to describe the combined processes of evaporation and transpiration from the surface of the soil and plant leaves, respectively, is measured by crop water requirement. The evaporation from the soil surface and the transpiration from plant stomatal activity are combined in the ET, a bio-physical process [1]. One of the most crucial elements of the hydrologic cycle and the water balance model is the process of determining ET. Numerous researches, including hydrologic water balance, crop yield simulation, irrigation system design and management, and planning and management of water resources, depend on its precise estimation.

An empirical crop coefficient (Kc) is used to moderate the estimated reference evapotranspiration (ETo) from a standard surface, which is a typical method for predicting ET at any given time for a particular crop. Even though a lysimeter is the optimal tool for measuring ETo, this strategy is not always feasible because to its time-consuming, expensive, and need for extensive fieldwork as well as careful observation. Because ETo data is widely used in agriculture and hydrology studies, over the past 70 years, a number of studies and research projects have focused on developing various types of mathematical models to indirectly estimate the ETo and enhance the performance of these models. The literature [2] offers a variety of indirect ETo estimation methods, from empirical to complicated models. However, the choice of appropriate models for ET estimation is contingent upon the data at hand, the features of the region, and the level of accuracy required. A sophisticated method known as the Penman-Monteith model [3,4,5,6] makes use of the physical properties of the ET process, such as heat and mass balance, by fusing energy and aerodynamic terms. This approach is acknowledged as the most effective worldwide method for estimating ETo in situations where measured ETo values are unavailable [7,8]. Moreover, a number of numerical models are calibrated and validated using P-M calculated ETo as the base method.

Because machine learning and artificial intelligence-based numerical approaches can map the input-output relationship without requiring a deep understanding of the underlying physical process, they have become increasingly used in evapotranspiration estimation over the past 20 years [9,10,11,12]. This has led to a reduction in complexity. The use of these machine learning techniques to modelling and evapotranspiration estimate has produced a number of success stories. With the introduction of the neural network model and its numerous variations, the use of advanced computational capabilities in the estimation of evapotranspiration was initiated [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28].

The ANN model further improved to the algorithm of support vector machine (SVM) which was (i) less complex structure, (ii) less data requirement for training, (iii) very fast in data training/learning (iv) parameter optimization with less risk of learning function’s local minimum unlike ANN and (v) less sensitive to initial randomization of the weight matrix. Thus, in the last decade, much of the focus shifted to the implementation of the SVM algorithm in modelling the evapotranspiration process [29]. But since 2015, simulating the evapotranspiration process with deep learning and extreme learning techniques has received a lot of interest. The decision tree method of the classification and regression tree (CART) type was primarily used in this technique [30,31,32,33,34]. However, by using the more reliable ensembled machine learning approach, the performance could be improved even more. The boosting techniques utilized in this ensembled machine learning approach further enhance the learning performance algorithm [35].

The focus of this study is on how different machine learning algorithms, including Random Forest, Gradient Boosting Machine (GBM), and Extreme Gradient Boosting Machine (XGBoost), perform in relation to one another. The application of these techniques and their comparative impact on the mapping ability of the evapotranspiration process is not been explicitly demonstrated in the literature [31,36,37,38,39,40,41,42,43]. Additionally, there are limited or no study has been carried out in the past to explore the generalization capabilities of these models based on machine learning approaches. This is particularly important because once these models can be generalized, they can be directly implemented in the location, where past data are unavailable to develop locally trained models. The current study summarizes the comparative effectiveness of these techniques in estimating the ETo for locations whose data are not included in the model training and validation, in addition to reporting the successful implementation of these enhancements in the modelling of the evapotranspiration process.

2. Materials and Methods

2.1. Location

The different locations selected for the present study represent the Karnataka state of India which mostly falls under Agro-Ecological Zone 10 called as Southern Plateau and Hill region in the Indian sub-continent (Figure 1). This region is spread to the extent of 11.30 ^oN to 18.30 ^oN and 74.0 ^oE to 74.30 ^oE and the area of the region is 19.20 million ha. The climate is typically semi-arid with 69% of the cultivable area comes under dryland agriculture. The major soil types include sandy loam, red calcareous and black soils. The region receives meagre annual rainfall ranging between 500-700 mm compelling inhabitant farmers (mostly small and marginal farmers owning less than 1 ha of culturable land) practicing sustenance farming due to low cropping intensity and agricultural productivity. Thus, water management at a regional scale assumes a key factor in sustainable and enhanced farm production in this region. The important crops grown in this region include Cotton, Jowar, Bajra, Groundnut, Millets, Banana, Turmeric, Onions, and Chilies along with fodder crops.

2.2. Data Set and Methodology

Figure 2 depicts the detailed methodology. For the meteorological station Bengaluru in the Indian state of Karnataka, daily meteorological data for fundamental climatic parameters such minimum and maximum temperature, minimum and maximum relative humidity, wind speed, solar radiation, and rainfall were gathered (Figure 1). Numerical values for Julian Days (1-365), months (1 to 12 respectively for January to December), and quarters (1 to 4 respectively for January-March; April-June; July-September and October-December) were also included in the input data set in order to capture the monthly and seasonal variation in the underlying evapotranspiration process. The chosen stations represent the Agro-climatic zone of the Southern Plateau and Hill in the Indian subcontinent. The 44 years of daily data correspond to all the basic parameters for the duration of January 01, 1976 to June 30, 2020 were used in the development and validation of models. As recommended by [5], a data quality check was carried out, and incorrect and missing records were excluded from the model training, validation, and testing phases. The data was divided into input data for validation (testing the model) and input data for learning (used for developing the model). Table 1 contains information about the different locations.

The models were developed using the climatic data of Bengaluru stations with corresponding PM ETo values as targets. For this purpose, daily records for the duration January 01, 1976 to December 31, 2017 excluding the incomplete and erroneous daily records were considered. For model testing, daily records from January 01, 2018 to June 30, 2020 were considered. The model thus developed was directly applied to 11 other locations that were not included in model development. In these locations, developed models were tested using the daily records from January 01, 2018 to June 30, 2020 as is the case with model validation using meteorological station Bengaluru. If discrepancies in any of the input parameters were observed, the entire daily record was not considered. Like datasets for model learning, for testing too, incomplete and erroneous records were not considered. Additionally, penman-monteith (P-M), which is explained below, was used to calculate daily potential evapotranspiration.

$$E{T}_o=\frac{0.408\Delta (R_n-G)+\gamma \frac{900}{(T_{mean}+273)}{u}_2(e_a-e_d)}{\Delta +{\gamma }^{*}}$$

(1)

where, ETo = Reference evapotranspiration (mm/d), $T_{mean}$= Mean of minimum and maximum temperature (^oC^-1), u₂ = Horizontal wind speed at 2-meter height, (m/s), $\Delta$= Slope of the saturation vapour pressure-temperature curve (kPa ^oC^-1), e_d = Saturation vapour pressure at air temperature (kPa),$R_n$= Net Radiation (MJ m^-2 d^-1), $\gamma$= Psychrometric constant (kPa ^oC^-1), $G$= Solar Heat flux density to the ground (MJ m^-2 d^-1), e_a = Saturation vapour pressure at air temperature (kPa).

2.3. Regression Tree Algorithm

The decision tree algorithm is a kind of supervised classifier that combines classification and regression analysis. The algorithm first classifies the target values in which a certain class is identified within which the target variable most likely falls. Later, the regression tree component or predictive model component of the decision tree predicts the value based on the classified target variable. This iterative process continues unless predefined criteria to terminate the process (usually subdivision process and minimizing the error) are met. The training data, which is initially included in the root node, is subdivided recursively during the creation of the regression tree model. The data is divided into branches, or subdomains, by the recursive subdivision. Most of them are linear regression models with many variables. The parent node of a branch is divided into left and right child nodes, or leaf nodes, as a result of further subdividing the data into branches. This process is also accomplished by a linear model, which is used to generate predictions.

The growth process of the regression tree is responsible for the recurrent recursive split of training datasets. This is accomplished by taking into account two factors: the subdivision process and minimising error, or impurity. In order to minimise the least square deviation, the division process is divided into smaller divisions at each stage. There are several variants of decision tree models based on the error optimization model used in the algorithm. The most adopted algorithm is Random Forest (RF) in which many decision trees are individually evaluated for the input and target datasets and then clubbed together (bagging) to find the global optimal solution. In another approach, the datasets are initially filtered through the function of the base learner before employing a decision tree algorithm for optimal solution. This approach is called boosting-type machine learning. Gradient Boosting Machine (GBM) and Extreme Gradient Boosting (XGBoost) are the most common algorithms under this approach.

2.4. Random Forest Algorithm (Bagging)—RF Model

Decision tree algorithms are best suited for the binary classification of data as they determine the optimal choice at each node and do not consider global optimization. The decision tree model leaves out bias-related error while attempting to reduce variance-related error. As a result, the decision tree algorithm may overfit, and problems with local minima may occur. The RF algorithm takes into account both variance and bias-related errors. The set of decision trees whose output is combined to get the final result is known as the random forest algorithm. This algorithm prevents significant bias-related inaccuracy by limiting overfitting. Because the random forest approach uses a random subset of data for training (row and feature subsampling), it lowers error due to variation. The final result is the culmination of the individual tree growth that each group of data produces. One can obtain comprehensive details regarding the RF model by referring [44].

2.5. Gradient Boosting Machine (GBM) Algorithm—GBM Model

Another type of ensembled machine learning method is the GBM algorithm [45]. This approach combines many base learners, which are basic functions, to produce a hypothesis function. A loss function was produced by the differentiation of the hypothesis function. Ultimately, the model learns as a result of the input of the training dataset and the loss function into GBM. Below is a brief explanation and presentation of the algorithm. The model is started with a constant function, $F_o\left(x\right)$, which is an optimisation problem, in the first phase.

$$F_o\left(x\right)={\gamma }_{optmal}=min\sum _{i=1}^{n}L\left(y_i,\gamma \right)$$

(2)

where, $L\left(y_i,\gamma \right)$ = loss function. Initially $F_o\left(x\right)$=$\gamma$ that fits the actual y-values in the data sets. The pseudo-residuals are then computed by differentiating the loss function, which is thus given as:

$$r_{im}=-\frac{\partial L\left(y, F_{m-1}\left(x\right)\right)}{\partial F_{m-1}\left(x\right)} , \mathrm{for} x=x_i, y=y_i \mathrm{and} i=1, 2,\dots ,n$$

(3)

The pseudo-residuals are used to replace $y_i$ in datasets with $r_{im}$. Pseudo-residual datasets are used for training and fitting a base learner, $h_m\left(x\right)$. The algorithm iteratively makes the following changes to itself until the termination conditions are satisfied.

$$F_m\left(x\right)=f_{m-1}\left(x\right)+{\gamma }_m{h}_m\left(x\right)$$

(4)

2.6. Extreme Gradient Boosting (XGBoost) Algorithm—XGBoost Model

An improvement to GBM is the ensembled machine learning algorithm XGBoost, which approximates the loss function using a Taylor series second-order approximation rather than pseudo-residuals as GBM does. The objective function, also known as the loss function, and the explicit regularisation term in the objective function are the two main components of the XGBoost algorithm. In a nutshell, the XGBoost model is explained like this:

$$L={\sum }_{i=1}^{n}l\left(y_i{y}_i^{t-1}+f_t\left(x_i\right)\right)+\mathsf{\Omega }\left(f_t\right)$$

(5)

$$\mathsf{\Omega }\left(f_t\right)=\gamma T+\frac{1}{2}\lambda {\omega }^2$$

(6)

where, $y_i$ and $y_i^{t-1}$= observed target and predicted target value respectively, $x_i$= input data which is approximated by the Taylor series, and $\mathsf{\Omega }$ = regularization function. The $\gamma$ and $\lambda$ = regularization terms which penalize T (the number of leaves) and $\omega$ (the weight of different leaves).

2.7. Performance Evaluation

The standard statistical evaluation criteria were adopted for the performance evaluation of developed models. Those include Mean Absolute Error (MAE), Average Absolute Relative Error (AARE), Coefficient of Correlation (r), Noise to Signal Ratio (NS), and Nash-Sutcliffe Efficiency ($ɳ).$ These evaluation equations are described in Table 2. The various developed models were evaluated in two stages. Firstly, the best-performing models were scrutinized using ɳ coefficient and MAE as defined in Table 2.

[7] evaluated several conventional ETo estimation models based on the statistical parameter of Weighted Standard Error of Estimate (WSEE). The WSEE is comprehensive as it exclusively considers the model error that occurred during peak season. This is the most important aspect as many irrigation systems are designed for water requirements in peak season. Therefore, the developed model was also tested for WSEE parameters. The steps for computing WSEE are given in the following paragraphs. Using the following equation, the standard error of estimate (SEE) for ETo estimated by the model for all months and peak months is computed. This shows the goodness of fit without any adjustments.

$$SEE={\left[\frac{{\displaystyle \sum _{i=1}^n{(E{T}_M-E{T}_{PM})}^2}}{n-1}\right]}^{0.5}$$

(7)

where ET_M = ETo estimated by model, ET_PM = ETo estimated by P-M, and n = overall count of data. The ET_PM and ET_M were used to fit the linear regression line as below.

$$E{T}_{PM}=b\times E{T}_M$$

(8)

where b = regression coefficient, used to adjust the ETo estimates and SEEs were recalculated for the adjusted SEE (ASEE) values. The WSEE is calculated as below [7]:

$$WSEE=0.7(0.67(SE{E}_{all})+0.33(ASE{E}_{all}))+0.3(0.67(SE{E}_{peak})+0.33(ASE{E}_{peak}))$$

(9)

where, all and peak = all months and peak month, respectively.

3. Results

3.1. Performance Evaluation of RF Model

The scatter diagram of the RF model estimated ETo and PM ETo is presented in Figure 3. For, all the locations, linear trends were observed albeit residuals are varied to some extent. The model is under estimating the higher values and over estimating the lower values. Thus, the range of model output shrinks as compare to observed values of PM ETo. Table 3 presents the performance statistics. The model could perform fairly across the location as the WSEE values are less than 1 mm/day except Ballari and Koppal where Nash Sutcliffe model efficiency and correlation coefficient deteriorated significantly. The model performed satisfactorily on other performance criteria as well. However, the performance deteriorated significantly in predicting higher values except for locations Bengaluru, Chikmaglur, and Mandya.

3.2. Performance Evaluation of GBM Model

Figure 4 presents the scatter plot between P-M ETo and those estimated using the GBM model for respective locations. Significant improvement in model performance was observed as compared to the RF model for all the parameters. In this case, the WSEE error was 0.87 mm/day for all the locations as compared to 1.05 mm/day for the RF model. The limitation of the RF model is that it underestimated higher values and overestimated lower values are addressed significantly in GBM model. This model also performed substantially better than the RF model on other selected model performance criteria (Table 4). The Nash-Sutcliffe model efficiency and correlation coefficient were more than 0.95 for all the locations. The lower noise to signal ratio indicated that the model output is less scattered and more robust as compared to the RF model.

3.3. Performance Evaluation of XGBoost Model

The XGBoost model further improves the performance in estimating the ETo to some extent over the GBM model. However, the improvement in model performance is significant as compared to the RF model like the case of the GBM model. In this case, lower values of NS could be obtained signifying more robustness of the model as compared to the GBM and RF model. This can be observed from the scatter plot presented in Figure 5 which is substantiated by higher correlation coefficient and Nash-Sutcliffe model efficiency as these values are more than 0.95 (Table 5). The WSEE in estimating PM ETo is slightly better than GBM model for all the locations as these are less than 0.85 mm/day.

4. Discussion

Over the past 20 years, there have been numerous reports in the literature about the effective use of sophisticated computer systems to estimate water requirements. On the other hand, not much has been reported on how well these computing methods generalise. All three of the chosen approaches in this study show some degree of generalisation capacity at varying degrees. Though it performs decently in the majority of locations, the more widely used RF model as a computational tool for calculating PM ETo lags noticeably behind GBM and XGBoost [46]. Both XGBoost and GBM make use of the fastest convergence technique when operating at par. For Bengaluru, the WSEE error was, nevertheless, as low as 0.19 mm/day.

This is to be expected, as model training and validation are done using this location data. The contrary finding relates to the generalisation capacity of the GBM and XGBoost models, which fared reasonably well for sites excluded from model validation and training. The use of temporal and spatial data to map the seasonal variations, such as quarter of the year, Julian day, longitude, and latitude may be one of the factors contributing to this outstanding performance. This is particularly significant because, in contrast to the PM method, which also indirectly takes into account the temporal and spatial data, most modelling work of this kind only takes into account the basic daily climatic data of six parameters: maximum and minimum temperature, sunshine hour or solar radiation, and minimum and maximum relative humidity, and wind speed. As a result, the modelling approach’s current framework more accurately reflects the underlying evapotranspiration mechanism as specified by the PM technique.

Due to its high degree of non-linearity, the ETo at any given day is not directly correlated with the weather from the day before. With little input data, the ensemble models were able to map the underlying nonlinear evapotranspiration process. Consequently, the findings imply that these models could be used to produce estimates of PM ETo that are reasonably accurate. Moreover, the outcome can be effectively applied in other locations due to the numerical models’ capacity for generalisation. Even though daily data were used in this study, hourly data can also be used using the same methodology. The weather station might be incorporated with the XGBoost model’s produced code.

5. Conclusions

Accurate quantification of ETo is necessary for different activities such as computation of the hydrological water balance, crop water requirement, crop yield simulation, irrigation scheduling, irrigation system design, reservoir operation, and water allocation. In this study, an attempt was made to develop generalized machine learning (ML) models to estimate ETo indirectly using climatic data. For this three ensemble ML models such as XGBoost, GBM, and RF were developed using the data from one location and tested on 11 different locations without any local calibration. The findings showed that the decision tree modelling approach based on ensembled machine learning algorithms effectively takes into consideration the nonlinear link between meteorological variables and related PM ETo. The performances are further enhanced when this strategy is accompanied by an ensembled machine learning algorithm, either in the form of GBM and XGBoost boosting or RF bagging approach. For each model and location, the WSEE values produced by the three models were less than 1 mm/day for PM ETo estimation. Though all the developed ML models performed better with respect to standard PM ETo method, XGBoost model estimated ETo more accurately as compared to GBM and RF models. In this work, the ensembled machine learning technique is used to develop a mathematical model for short-term ETo estimation. This is especially crucial for real-time water management when estimating PM ETo. The "on-demand" irrigation water supply made possible by the ETo estimation, which has the potential to improve overall irrigation efficiency in the command area of large irrigation projects.