Flood Forecasting Method and Application based on Informer Model

1. Introduction

Our country is one of the countries with frequent flood disasters, different types and different degrees of floods may happen on about two-thirds of the land area. According to the statistics of the National Center for Disaster Reduction under the Ministry of Water Resources and the Ministry of Emergency Management, from 1991 to 2020, the number of people killed or missing due to floods in China reached 2,020, with a total of more than 60,000 deaths. Therefore, we need to predict the flood flow of the reservoir according to the information of rainfall and flow in the upstream of the basin through time series prediction, so as to give early warning and try to avoid safety accidents caused by sudden flood peaks [1,2]. In the past, hydrological models were generally used, but these models required a lot of parameters, such as temperature, soil moisture, soil type, slope, terrain, etc., and different parameters also contained very complex relationships [3]. In recent years, machine learning technology has developed rapidly, and many researchers have found that its efficient data parallel processing ability can be applied to the field of flood prediction [4,5].

The Artificial Neural Networks (ANN), one of the data-driven techniques, have been widely used inhydrology as an alternative to physical-based and conceptual models [6,7]. Liu Heng et al. [8] proposed a flood classification forecast model, which greatly improved the forecast accuracy. Li Dayang et al. [9] proposed the VBLSTM model combining Stochastic Variational Inference (SVI) and Long Short-Term Memory (LSTM), and experiments found that the prediction accuracy is higher than that of traditional hydrological models. XU et al. [10] proposed to use Particle Swarm Optimization (PSO) to optimize LSTM hyperparameters for flood prediction tasks in Fenhe Jingle Basin and Luohe Luoshi Basin, improving the accuracy of LSTM model for flood prediction. Cui Zhen et al. [11] constructed a mixed model of GR4J (modele du Genie Rurala4 parametres Jour⁃nalier) and LSTM based on the conceptual hydrological model to forecast the inflow of land water reservoir. The results show that GR4J-LSTM has better flood forecasting performance than a single model. Compared with traditional hydrological models and shallow machine learning models [12,13,14], machine learning models show superior performance in flood prediction.

The Transformer model [15] proposed in 2017 supports parallel computing, is faster to train, and can simultaneously model long-term and short-term dependencies, and has shown good results in processing temporal data series [16,17]. In order to better realize time series prediction under different tasks, scholars in related fields have improved it in many ways. For example, The MCST-Transformer [18] (Multi-channel spatiotemporal Transformer) is used to predict the traffic flow, and the XGB-Transformer (Gradi⁃ent Boosting Decision) is used Tree transformer model [19] has been used for power load prediction, solving the problem of Transformer model being insensitive to local information in time series prediction tasks [20], and Transformer-based dual encoder model for the prediction of monthly runoff of the Yangtze River [21].

However, Transformers have three problems: high computational complexity, large memory usage, and low prediction efficiency.In 2021, Zhou [22] from Beihang University proposed the Informer model based on the classic Transformer encoder-decoder structure solve these problems to some extent. In this study, the data collection interval was long, and the Informer model was used for prediction and compared with other models.

2. Methodology

2.1. Development of the Approach

Figure 1 shows the process of the reservoir flood discharge prediction method. Firstly, the flood discharge data of Wan ’an Reservoir over the years were collected and pre-processed, including dividing the data into a training set (70%) and a prediction set (30%). Then the data data normalization is carried out, and the flood flow prediction is realized by using the three models of ANN, LSTM and Informer. NSE,RMSE,R2 and RE were calculated as evaluation criteria according to the flood flow prediction results. At the same time, the number of flood fields with flood peak gap less than 15%, total flood gap 15%, NSE greater than 0.8, and the maximum flood peak difference were counted as reference.

2.2. Deep Learning Models

2.2.1. ANN

Artificial Neural Network (ANN) is a powerful tool for dealing with machine learning problems in the computer field. It is widely used in regression and classification problems. It simulates the operation principle of biological nerve cells, and forms a network structure of artificial neurons with hierarchical relationship and connection relationship. By means of mathematical expression, the signal transmission between neurons can be simulated, so as to establish a nonlinear equation with input and output relationship, and can be visualized through the network, we call it artificial neural network. Generally speaking, ANN can fit any nonlinear function through reasonable network structure configuration, so it can also be used to deal with nonlinear systems or black box models with complex internal expression.

2.2.2. Long Short-Term Memory

The LSTM network is a modified recurrent neural network proposed by Hochreiter and Schmidhuber. In recent years, the research on sequence prediction problem mainly focuses on the prediction of short sequences. LSTM network is used to conduct experiments on a set of data for short time series (12 data points, 0.5 days of data) and long time series (480 data points, 20 days of data). The results show that with the increase of sequence length, the prediction error increases significantly, and the prediction speed decreases sharply. And LSTM models have several serious problems:High prediction error: When dealing with long time series data, the prediction error of LSTM model is high, which makes it perform unsatisfactory in some application scenarios;Slow prediction speed: The LSTM model has a relatively slow prediction speed, which is mainly caused by its internal complex calculation process; More model parameters: The LSTM model has more parameters to train, which makes it require more computing resources and time when dealing with large-scale data; Prone to mode switching: When dealing with non-stationary time series data, the LSTM model is prone to mode switching, which will lead to instability of the model prediction results.

2.2.3. Informer

Recent studies have shown that compared with RNN type models, Transformers show high potential in the expression of long-distance dependencies. However, Transformer has the following three problems: the quadratic computational complexity of self-attention mechanism: the time complexity and memory usage of each layer due to the dot product operation of self-attention mechanism; High memory usage problem: When stacking long sequence inputs, the stack of J encoder-decoder layers makes the total memory usage as, which limits the scalability of the model when accepting long sequence inputs. Efficiency in predicting long-term outputs: The dynamic decoding process of the Transformer, where the output becomes one after another, and the subsequent output depends on the prediction of the previous time step, results in very slow inference.

The authors of Informer target Transformer models with the following goal: Can Transformer models be improved to be more computationally, memory, and architecture-efficient while maintaining higher predictive power? To achieve these goals, this paper designs an improved Transformer based LSTF model, namely Informer model, which has three notable features: a ProbSpare self-attention mechanism, which can achieve a low degree of time complexity and memory usage; In the self-attention distillation mechanism, a Conv1D is set on the results of each attention layer, and a Maxpooling layer is added to halve the output of each layer to highlight the dominant attention, and effectively deal with too long input sequences. The parallel generative decoder mechanism outputs all prediction results for a long time sequence instead of predicting in a stepwise manner, which greatly improves the inference speed of long sequence prediction in Figure 2.

The model belongs to the Encoder-Decoder structure, in which the self-attention distillation mechanism is located in the Encoder layer. This operation essentially consists of several encoders and aims to extract stable long-term features. The network depth of each Encoder gradually decreases, and the length of the input data also decreases, and finally the features extracted by all encoders are concatenated. It should be noted that in order to distinguish encoders from each other, the authors gradually decrease the depth of each branch by determining the number of repetitions of the self-attention mechanism of each branch. At the same time, in order to ensure the size of the data to be merged, each branch only takes the second half of the input value of the previous branch as input.

The self-attention distillation mechanism is an efficient method to improve the efficiency and accuracy of the model by training a smaller model to guide the learning of a larger model. The core idea of the self-attention distillation mechanism is to use the attention distribution of the large model to guide the training of the small model when training the small model, so that the small model can better imitate the large model.

The principle of the self-attention distillation mechanism is that it captures the global characteristics of the input data by computing the attention distribution of the input data. Then, these features are passed to the small model so that the small model can better imitate the large model.

The generative Decoder is located in the Decoder layer. In the past time series prediction, it is often necessary to perform multi-step prediction separately to obtain the prediction results of several future time points. However, with the continuous increase of the prediction length, the cumulative error will become larger and larger, resulting in the lack of practical significance of long-term prediction. In this paper, the authors propose a generative decoder to obtain the sequence output, which only requires a single derivation process to obtain the prediction result of the desired target length, effectively avoiding the accumulation of error diffusion during multi-step prediction.

Another innovation is the probabilistic sparse self-attention mechanism, which is proposed from the author’s thinking about the feature map of the self-attention mechanism. The authors visualize Head1 and Head7 of the first layer of self-attention mechanism, and find that there are only a few bright stripes in the feature map. At the same time, a small part of the scores of the two heads have large values, which is consistent with the distribution characteristics of long-tailed data, as shown in the figure above. The conclusion is that a small fraction of the dot product pairs contribute the main attention, while the others can be ignored. According to this characteristic, the authors focus on the high-scoring dot product pairs, trying to calculate only the high-scoring parts in each operation of the self-attention module, so as to effectively reduce the time and space cost of the model. It allows the model to automatically capture the relationship between the individual elements of the input sequence, so as to better understand the input sequence. This mechanism enables the model to process all elements of the input sequence in parallel, avoiding the problem of computational order dependence in sequence models such as RNN/LSTM, and thus processing more efficiently.

2.3. Model Application

The water flow data of Wan ’an Reservoir is used as the data set for prediction. The dam site of Wan ’an Reservoir is located 2 kilometers upstream of Furong Town, Wan ’an County, Jiangxi Province, at 114°41 ’east longitude and 26°33’ north latitude. The upstream is 90 kilometers away from Ganzhou City and the downstream is 90 kilometers away from Ji ’an City, and the control basin area is 36900 square kilometers, seeing Figure 3.

There are several hydrological stations and rainfall stations in the basin of Wan ’an Reservoir. In this paper, the rainfall information of five regions around the reservoir (PQJ1, PQJ2, PQJ3, PQJ4, PQJ5) and the flow information of Xishan hydrological Station and Julongtan hydrological station are selected to predict the flow of Wan ’an Reservoir. Rainfall and discharge data were collected in the region from May 17, 2014 to June 14, 2020, and the collection interval was 6 hours. See Figure 4.

Figure 4. Dataset of Wan’an reservoir.

Figure 4. Dataset of Wan’an reservoir.

In the above figure, FDNO is the flood number, TM is the record date, PQJ1, PQJ2, PQJ3, PQJ4, and PQJ5 are the rainfall information of the five rainfall stations in the reservoir area, xiashan and julongtan are the discharge information of the two discharge stations in the reservoir area, and OT is the actual discharge of Wan ’an Reservoir.

Due to the noise and jitter of the initial data, the smooth function is used to smooth the data, and the value at time t after processing is the average value at time t-1, t and t+1. In addition, the flow at the flood peak needs to be recorded, so the flow at the flood peak is not smoothed.

In this experiment, considering the long sampling time, the step size is set to 4, 5, 6, and the output length is 1, respectively. When the step size is 4, four data are used to predict one data in the future.

The commonly used Loss functions in machine learning regression problems include mean absolute error (MAE), mean square error (MSE) and Huber Loss. MSE loss usually converges faster than MAE, but MAE loss is more robust to outliers, that is, less susceptible to outliers. Huber Loss is a Loss function that combines MSE and MAE and takes the advantages of both, also known as Smooth Mean Absolute Error Loss. The principle is simple: MSE is used when the error is close to 0 and MAE is used when the error is large.

In this paper, more attention is paid to the peak flood flow since it has the greatest impact on reality, and MAE, MSE and Huber Loss functions are used in this experiment to compare the prediction effect.

In the predictions of the two models, the common parameters are learning rate, training epochs, patience and batch_size, respectively 0.0005,100,10,10.

All numerical experiments in this study were implemented on a Windows system (CPU: Intel i7-12700H, GPU: NVIDIA GeForce RTX 3070 using Python (3.9) based on the Pytorch (1.8.0).

2.4. Model Performance Measures

Four indicators are finally used to measure the prediction results, namely NSE (Nash coefficient), R² (determination coefficient), RMSE (mean square error), and RE (mean difference).

Four hydrological concepts were used to measure the prediction results, namely, the number of fields with the peak discharge gap less than 0.15, the number of fields with the total flood gap less than 0.15, the number of fields with the Nash coefficient greater than 0.8 and the maximum flood peak gap.The calculation equations are as follows:

$$\begin{array}{c}\mathrm{NSE}=1-\frac{{\sum }_{i=1}^n{\left(y_i-x_i\right)}^2}{{\sum }_{i-1}^n{\left(x_i-\overline{x}\right)}^2}\end{array}$$

(1)

$$\begin{array}{c}{\mathrm{R}}^2=\frac{{\left[{\sum }_{i=1}^n\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)\right]}^2}{{{\sum }_{i=1}^n\left(x_i-\overline{x}\right)}^2{{\sum }_{i=1}^n\left(y_i-\overline{y}\right)}^2}\end{array}$$

(2)

$$\begin{array}{c}\mathrm{RMSE}=\sqrt{\frac{{\sum }_{i=1}^n{\left(y_i-x_i\right)}^2}{n}}\end{array}$$

(3)

$$\begin{array}{c}\mathrm{RE}=\frac{{\sum }_{i=1}^n{y}_i-{\sum }_{i=1}^n{x}_i}{{\sum }_{i=1}^n{x}_i}\end{array}$$

(4)

where x_i and y_i are observed and simulated values, respectively; x and y are average observed and simulated values, respectively and n is the number of samples.

NSE (Nash coefficient): It is used to verify the quality of hydrological model simulation results. The value of NSE is negative infinity to 1, and NSE is close to 1, indicating that the model quality is good.

R² (Coefficient of Determination): The proportion that reflects the total variation of the dependent variable can be explained by the independent variable through the regression relationship. It ranges from 0 to 1. The greater the coefficient of determination, the better the prediction effect.

RMSE (mean square error): It is the square root of the ratio between the squared deviation of the predicted value from the true value and the number of observations. It tells you how discrete a dataset is.

RE (mean difference): It is the ratio of the absolute error caused by a measurement to the measured (agreed) true value multiplied by 100%, which reflects the confidence of the measurement.

Peak discharge: the maximum instantaneous discharge in a flood discharge process, that is, the highest discharge on the flood process line. It may be the measured value, or it may be the calculated value using the water-flow relationship curve or the calculation value of the hydrodynamic formula.

Total flood water: The total amount of flood water flowing from the outlet section of the basin in a certain period of time. The total amount of a flood caused by a rainfall is often calculated in the forecast of rainfall runoff, which can be obtained from the area between the beginning time of the flood flow and the end time on the retreating section of the flood process line.

3. Result

Using thirty percent of the data as the prediction set, the different seq lengths and prediction accuracy metrics based on the loss types (MAE, MSE, and Huber) of the three deep learning models (ANN, LSTM, and Informer) are shown in Table 1.

In addition, considering that this study is application-oriented, the trained model is used to predict all data sets, and the results are obtained and compared with the original data, as shown in Table 2.

The number of the following four indicators are shown in Table 2: difference values less than 15%, flood peak difference less than 15%, NSE more than 0.8, max flood peak gap.

4. Discussion

In the Table 1 and Table 2, the best prediction results of the three models are marked in red, and the best prediction results are based on the statistical results of Informer. The result with the highest NSE (Nash coefficient) is selected, that is, the step size is 4 and the loss function is MAE. Two floods (No. 1 and No. 2) are selected to draw the rainfall flow diagram, see Figure 4 and Figure 5.

Figure 4. No. 1 Flood rainfall flow chart(Informer).

Figure 4. No. 1 Flood rainfall flow chart(Informer).

Figure 5. No. 2 Flood rainfall flow chart(Informer).

Figure 5. No. 2 Flood rainfall flow chart(Informer).

From Figure 4 and Figure 5, it can be found that the predicted flood process line has a high degree of agreement with the actual flood process, and compared with the flood peak of Xiashan and Julongtan, the flood peak is lagged behind, which better reflects the spatiotemporal information of the flood.

According to Table 1, 30% of the data is taken as the prediction set, and the average value of the nine experimental results is taken as the result to draw the graph, as shown in Figure 6, where the horizontal axis is the four prediction indicators, and the vertical axis is the average value of the experimental results after appropriate processing.

Figure 6. Prediction results indicator bar chart(prediction set).

Figure 6. Prediction results indicator bar chart(prediction set).

According to Table 2, all data sets are used for prediction, and the bar charts of the four indicators are shown in Figure 7.

Figure 7. Prediction results indicator bar chart.

Figure 7. Prediction results indicator bar chart.

According to Table 3, for each hydrological index, the average value of the nine experimental results is taken as the result to draw a diagram, as shown in Figure 8, where the horizontal axis is the four hydrological indicators(A:difference values less than 15%, B:flood peak difference less than 15%, C:NSE more than 0.8, D: reciprocal of max flood peak gap) and the vertical axis is the average number.

Figure 8. Histogram of hydrographic indicators.

Figure 8. Histogram of hydrographic indicators.

5. Conclusions

In this study,Informer model and traditional hydrology methods were combined to study the effect of ANN,LSTM and Informer models in predicting the flood discharge of Wan ’an Reservoir, and NSE, RMSE, R² and RE were used to evaluate the accuracy and reliability of the prediction, all of which showed satisfactory results. Therefore, using machine learning method to predict flood flow is reliable. The results show that Informer is superior to ANN and LSTM in most cases.

The R² of LSTM model is better than Informer model except for 3 cases (loss=MAE, seq len=4),(loss= MAE, seq len=5),(loss=MSE, seq len=4). In other cases, the statistical indicators of the Informer model are better than the other two models. However, when predicting with the full dataset, the R² of the LSTM model (loss= MAE, seq len=5) is better than that of the Informer model in both cases (loss=MSE, seq len=6). In one case (loss=Huber, seq len=5), the LSTM model has better RE than the Informer model, and in the remaining cases, the Informer model predicts the best result. In terms of hydrological indicators, LSTM outperforms Informer in three cases, ANN outperforms Informer in four cases, and Informer predicts better in the remaining cases. Therefore, we can conclude that Informer performs better than ANN and LSTM models in predicting Wan ’an reservoir.

According to Figure 6, Figure 7 and Figure 8, we can get the final conclusion that the average values of the four indicators (NSE, R², RMSE, RE) of the Informer model are better than those of ANN and LSTM under nine different conditions.