Forecasting Lake Nokoué Water Levels Using Long Short-Term Memory Network

2. Materials and Methods

2.1. Study Area

Nokoué lake, (Figure 1), is located in the southeast of Benin, between 6°25'N and 2°36'E, covering an area that varies between 150 km² and 170 km² respectively during the low water period and high-water period, respectively [26,27,28,29]. It stretches for approximately 20 km from east to west along the coast and 11 km from south to north, as confirmed by multiple authors such as[27,28]. The average and maximum depths of the lake are approximately 1.3m and 2.9m, respectively. Towards the Cotonou channel, the Nokoué lake deepens, and the average and maximum depths reach around 3m and 8m, respectively. Two rivers flow into Nokoué lake: on its northern bank is the Sô-Ava river, which drains a watershed area of approximately 10,000 km², and the Ouémé river, the largest river in Benin, which drains a watershed area of approximately 50,000 square kilometer [26,28]. The Djonou river, with a smaller extent and flow, also contributes to the freshwater input in the southwestern part of Nokoué lake. In the southern part, Nokoué lake is connected to the Atlantic Ocean through the Cotonou channel, which is 280 meters wide and approximately 4 kilometers long [26,29]. Through this canal, constructed in 1885, exchanges of freshwater and saltwater occur in accordance with the tides and hydrological regime [1]. The canal of Tochè, approximately 4 km long, connects the Porto Novo lagoon, with an area of approximately 35 km², to Lake Nokoué on the eastern side, with little effect on the dynamics of Lake Nokoué. At a seasonal scale, the hydrological regime of Lake Nokoué is determined by the West African summer monsoon, resulting in two rainy seasons and two dry seasons. [1]. These seasons are linked to the north-south movement of the intertropical convergence zone and the associated belt of intense tropical rainfall. The main rainy season extends from April-May to the end of July when the intertropical convergence zone moves northward from its southern position near the equator. The second rainy season, shorter and less intense, occurs from late September to November when the intertropical convergence zone migrates southward from its northernmost position. However, this dual rainy season, along with local precipitation, has only a weak influence on the water level of Nokoué lake. Nokoué lake is more influenced by the hydrology of the central part of Benin, where the main basin of the Ouémé river is located [29]. This region is characterized by a single rainy season with peak precipitation occurring between July and October [29,30]. The period from September to October is when the maximum flow enters Nokoué lake.

2.2. Data Acquisition

The data used in this study include a times series of daily water level observations expressing flood-recession events, a series of rainfall observations, and a series of discharge observations provided by the National Meteorological Agency of Benin (METEO-Benin) and the General Directorate of Water of Benin (DGEau).

2.3. Data Preprocessing

Preprocessing of the variables is necessary to account for the full range of values (both low and high) and to avoid sigmoid saturation with the high values in the database [31]. Indeed, the direct application of the sigmoid function to the weighted sums of rainfall-discharge inputs results in the neglect of information from low numerical values (rainfall and water levels) compared to high numerical values (discharge). This preprocessing step involves normalizing all values in the database between -1 and 1. This is done through the following Equation 1:

$${\mathrm{X}}_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{z}\mathrm{e}\mathrm{d}}=\frac{{\mathrm{X}}_i-{\mathrm{X}}_{mean}}{\mathrm{s}\mathrm{t}\mathrm{d}}$$

(1)

where X_i is the actual value to be normalized, X_mean is its mean, and X_{normalized​} is the normalized value. This transformation scales the input data between [-1, 1].

2.4. Structure of the Long Short Term Memory Model

The Long Short-Term Memory (LSTM) cell is an enhancement of the optimization proposed in [32]. Considered as a black box, the LSTM cell, widely used in time series forecasting, is an impressive architecture of an artificial recurrent neural network (RNN) capable of memorizing the temporal order of data. Furthermore, the LSTM overcomes the issues of gradient instability and insufficient memory capacity through the state of its cells and gates [33]. The two main problems of the RNN architecture are gradient instability and its inability to retain information from long sequences of temporal data. In contrast, the LSTM cell, as a deep learning predictive model, receives the latent states from the previous step and has a self-evaluation mechanism that offers better performance in time series forecasting. The internal structure of the LSTM consists of three main gates that control the flow of information: (i) forgetting unwanted information in the current cell state through the forget gate (f_t), (ii) adding additional data to the current cell state through the input gate, also known as the temporal attention module (I_t), and (iii) producing an output from the current cell state through the output gate (O_t). These gates serve specific operations on the cell states. The state of the LSTM network is divided into two states: h_t and c_t. The hidden state h_t of the LSTM network is considered as short-term memory, while the cell state c_t is considered as long-term memory of the network. The operations performed within the LSTM cells help the model retain information from sequential data. The LSTM network uses cells as memory units for the model. The gates, as shown in Figure 2 and illustrated in Equations 1, 4, 5, and 6, determine the data to be carried.

a)

The forget gate

The forget gate, f_t, determines the amount of information from the previous timestamp that should be transmitted and is the essence of the LSTM architecture. It determines the amount of memory preserved from the previous memory state, C_t-1. In Equation 2, the previous hidden state, h_t-1, and the current input information, x_t, are passed through the sigmoid activation function. Information associated with 0 is forgotten, while information associated with one 1 continues to be carried through the cell state.

$$f_t=\sigma \left(w_{fh}\times h_{t-1}+w_{fx}{x}_t+b_f\right)$$

(2)

Where:

σ is the sigmoid activation function ;

w_fh and w_fx represent the weight matrices of the forget gate, f_t, for their connections to the previous hidden state h_t–1 and to the input vector x_t;

b_f the bias term for the forget gate.

• b) Long-term state

The flow of the long-term state, c_t-1, through the network is from left to right. It first passes through a forget gate, which discards certain information, and then adds new information through an addition operation (the added information is selected by an input gate), and the resulting c_t is sent directly without any further transformation. Therefore, at each time step, information is removed and new information is added. The update calculation of c_t is illustrated in the following Figure 2 and Equations 3 et 4:

$$g_t=\mathit{tanh}\left(w_{gh}\times h_{t-1}+w_{gx}{\times x}_t+b_g\right)$$

(3)

$$c_t=f_t\times c_{t-1}+I_t\times g_t$$

(4)

Where :

w_gh and w_gx represent the weight matrices of the main layer g_t for their connections to the previous short-term state h_t–1 and to the input vector x_t;

b_g is the bias term for the main layer.

• c) The input gate

The input gate, It, in Equation 5 determines which parts of the main layer, g_t, to update in the long-term state. The previous and current information is updated based on the result of the sigmoid operation (σ). Information associated with 0 is considered trivial, while information associated with 1 is deemed essential. Additionally, the hyperbolic tangent activation function (tanh), which compresses data between -1 and 1, is used to regulate the network. The outputs of the sigmoid and tanh functions are then multiplied to select the information that will be updated.

$$I_t=\sigma \left(w_{ih}\times h_{t-1}+{w_{ix}\times x}_t+b_i\right)$$

(5)

Where :

w_ih and w_ix represent the weight matrices of the input gate for their connections to the short-term state h_t-1 and to the input vector x_t ;

b_i the bias term of the input gate.

• d) The output gate

In Equations 6 and 7, the output gate, o_t, is also used for inference and determines which parts of the long-term state should be read and output at this time step, both in h_t and in y_t. Additionally, after the addition operation, the long-term state is copied and passed through the hyperbolic tangent function (tanh), with the result being filtered by the output gate. The result is the short-term state, ht, which is equal to the output y_t of the LSTM cell.

$$o_t=\sigma \left(w_{oh}\times h_{t-1}+{w_{ox}\times x}_t+b_o\right)$$

(6)

$${y_t=h}_t=o_t\times \mathit{tanh}\left(c_t\right)$$

(7)

Where:

w_oh and w_ox represent the weight matrices of the output gate for their connections to the short-term state h_t-1 and to the input vector x_t;

b_o is the bias term of the output gate;

h_t is the output result of the masked layer at time step t.

2.5. Long Short-Term Memory Model Configuration

In this work, optimizing the performance of the LSTM model involves selecting the input variables, determining the appropriate network architecture, optimizing network learning, and using a reliable validation methodology. The LSTM network, as explained earlier, consists of an input layer, a single hidden layer, and an output layer with sigmoid activation functions for the artificial neurons and hyperbolic tangent for the hidden states. The optimal initialization of the model's learning algorithm parameters, such as the number of hidden neurons, the optimization function, the number of iterations, and the batch size, for performance estimation is performed using the random search cross-validation method (randomSearchCV), during which we test and evaluate different combinations of inputs (rainfall, discharge, water level). The preprocessed database is divided into two parts:

– the part intended for training to recognize the system's dynamics, which is the most important part (80%);

– the testing part (20%) which prevents overfitting by checking and testing the loss function evolution during training and validation. After the training is stopped and the weights of the interconnections of the most performing model are saved. The validation dataset allows for confirmation of the LSTM model's performance.

2.6. Model Performance Assessment

To ensure synchronization between the observed flood and the one estimated by the LSTM model, we rely on a qualitative evaluation of the LSTM model using various evaluation criteria. There is a wide range of performance evaluation criteria for hydrological models proposed by the World Meteorological Organization and other authors. [9,31,32,34]. To ensure the reliability of the LSTM model results in this study, we used four most relevant metrics (equations 8 to 11). These criteria include the Nash criterion, the Root Mean Square Error (RMSE), the Mean Absolute Error (MAE), and the coefficient of determination (R²).

$$NASH=1-\frac{{\sum }_{i=1}^N{\left(Q_i-{Q\text{'}}_i\right)}^2}{\sum _{i=1}^N{\left(Q_i-\overline{Q}\right)}^2}$$

(8)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\sum }_{i=1}^n\frac{{\left(Q_{obs}^i-Q_{sim}^i\right)}^2}{n}}$$

(9)

$$\mathrm{M}\mathrm{A}\mathrm{E}=MAE=\frac{1}{n}{\sum }_{i=1}^n\left|Q_{obs}^i-{\overline{Q}}_{sim}^i\right|$$

(10)

$$R^2=1-\frac{{\sum }_{i=1}^n{\left(Q_{obs}^i-Q_{sim}^i\right)}^2}{{\sum }_{i=1}^n{\left(Q_{obs}^i-{\overline{Q}}_{obs}\right)}^2}$$

(11)

3.1. Variable Selection and Statistics

3.1.1. Selection of the Variables

The variables selected for the model following forward feature selection are presented in Figure 1 and Figure 2. Rainfall, discharge of the Ouémé river and water level in lake Nokoue were the predominant predisposing factors in the lake Nokoue. Rainfall and discharge were selected as input for the model and the output was the water level of lake Nokoue.

Figure 3. Water level of lake Nokoue (Output variable).

Figure 3. Water level of lake Nokoue (Output variable).

Figure 4. Rainfall and discharge (selected input variables).

Figure 4. Rainfall and discharge (selected input variables).

3.1.2. Statistics of the Variable

The statistical results of the variables are explained in Table 2. These values of discharge variable were the among values, presenting mean and max values above 219.113 and 1064.000, respectively. The variation in the water level of lake Nokoué is strongly influenced by rainfall [37,38].

Table 1. Statistics of the all variables.

statistics	rainfall	discharge	Water level
mean	3.392	219.113	3.173
std	10.852	318.670	0.195
min	0.000	0.610	2.736
25%	0.000	8.225	3.045
50%	0.000	25.590	3.115
75%	0.200	376.900	3.257
max	158.200	1064.000	3.981

Table 1. Statistics of the all variables.

statistics	rainfall	discharge	Water level
mean	3.392	219.113	3.173
std	10.852	318.670	0.195
min	0.000	0.610	2.736
25%	0.000	8.225	3.045
50%	0.000	25.590	3.115
75%	0.200	376.900	3.257
max	158.200	1064.000	3.981

3.2. Model Performance Analysis

The evaluation of a model’s performance is an integral step in modeling. In this study, the RMSE, NSE, R² and MAE were used to assess model accuracy. The resulting RMSE, NSE, R² and MAE scores for the LSTM model are shown in Table 2 for different forecast horizons. Depending on the prediction horizon from t+1 day to t+10 days, the values of all performance criteria show a nearly stable pattern during both phases. The values of all performance criteria remain constant for the forecast horizons of t+1 days, t+2 days, and t+3 days (Table 2). Additionally, the R² value is greater than 0.97 in the calibration phase and greater than 0.96 in the validation phase. These satisfactory results demonstrate that the LSTM model performs well in both periods (calibration phase, also known as the learning phase, and validation phase, also known as the testing phase). It is capable of reliably reproducing the observed water levels in the Nokoué lake basin up to a forecast horizon of t+10 days. This enables significant proactive anticipation of flood events.

Table 2. Performance metrics values for different Prediction Horizons .

Forecast horizon		Training step				Testing step
	RMSE	NSE	R²	MAE	RMSE	NSE	R²	MAE
t+1 day t+2 days t+3 days t+4 days t+5 days t+10 days	0.03 0.03 0.03 0.03 0.03 0.03	0.98 0.98 0.98 0.94 0.98 0.92	0.98 0.98 0.98 0.98 0.98 0.97	0.02 0.02 0.02 0.02 0.02 0.02	0.04 0.04 0.04 0.03 0.03 0.04	0.97 0.97 0.97 0.96 0.97 0.90	0.97 0.97 0.97 0.97 0.97 0.96	0.03 0.03 0.02 0.02 0.02 0.03

Table 2. Performance metrics values for different Prediction Horizons .

Forecast horizon		Training step				Testing step
	RMSE	NSE	R²	MAE	RMSE	NSE	R²	MAE
t+1 day t+2 days t+3 days t+4 days t+5 days t+10 days	0.03 0.03 0.03 0.03 0.03 0.03	0.98 0.98 0.98 0.94 0.98 0.92	0.98 0.98 0.98 0.98 0.98 0.97	0.02 0.02 0.02 0.02 0.02 0.02	0.04 0.04 0.04 0.03 0.03 0.04	0.97 0.97 0.97 0.96 0.97 0.90	0.97 0.97 0.97 0.97 0.97 0.96	0.03 0.03 0.02 0.02 0.02 0.03

The adaptability of LSTM to the Nokoué lake basin during the calibration and validation periods is also supported by the convergence of the loss function evolution during both phases, as shown in Figure 5. Corresponding, the results from Ling et al.[39] showed that the method proposed in this article was more effective than other methods of artificial recurrent neural networks. The loss function values decrease with the number of iterations, stabilizing below 0.1, indicating maximum optimization of the model's performance during the calibration and validation phases. Given the architecture of the LSTM model, it was noted in the study by Hu et al. [40] a tendency of the LSTM model towards a local optimum. The hydrographs of observed and predicted water levels by the LSTM model during the calibration and validation phases are depicted in Figure 6a and 6b. It is evident that the observed and predicted floods and recessions are nearly identical. This synchronization between observed and estimated floods demonstrates the LSTM model's ability to reproduce critical water levels that could lead to flooding. Recently, a number of existing literature studies have been considered the classification of streamflow forecasting using recurrent neural networks models. Thapa et al. [41] developed a deep learning long-short-term memory (LSTM)-based model in the Himalayan basin for snowmelt-based discharge modeling. Ni et al. [42] applied the deep learning method for daily flow simulation, and used data from previous years for flow prediction. The model was carried out according to several perspectives. At the end of the study, it was found that the LSTM model was advantageous in processing constant flow data in the dry season and gave satisfying results in capturing data features in rapidly fluctuating flow data in rainy seasons. Luo et al. [43] built a new hybrid model based on the long-short-term memory approach for predicting streamflow. In this study, the linear regression model, which is one of the classical methods, was used to show how successful the performance between the benchmark model and the hybrid model was. The satisfactory forecasting results are further evident in Figure 7a, 7b, Figure 8a, and 8b, which present scatter plots of predicted (estimated) water levels against observed water levels. The linear trend line of the scatter plots during the training (calibration) and testing (validation) phases highlights the strong correlation between observed and predicted water levels, with a correlation coefficient of 0.98 during calibration and 0.97 during validation (Table 2). This linear alignment, particularly for water levels between 3.5 meters and 3.9 meters, indicates that the LSTM model accurately predicts extreme water levels in Nokoué lake that could lead to flooding. Figure 7b and Figure 8b illustrate a good distribution of the LSTM model's residual errors, with values ranging from -0.1 to 0.4 during the training phase and -0.5 to 0.1 during the testing phase. Negative residual errors indicate overestimation of water levels, while positive residual errors indicate underestimation of water levels in Nokoué lake by the LSTM model. These underestimations and overestimations remain acceptable for predicting extreme water levels in Nokoué lake.