LSTCN Approaches for Self-Adaptive Real-Time Weather Forecasting

1. Introduction

Predicting the future is one of humanity’s age-old dreams [29]. Science therefore aims to predict, but also to understand, the phenomenon observed, using an explanatory model [2,3,29] . The phenomenon chosen for our study is meteorology.

Meteorology is the interdisciplinary science of atmospheric physics, which studies weather conditions, the atmospheric environment, the phenomena produced, and the laws that govern it [4]. The process of meteorological measurement is directly linked to the establishment of a meteorological station [4,5,6,7]. In addition, a meteorological station is a set of sensors that record and provide physical measurements and meteorological parameters related to climate variations in a locality [4,5,7]. These collected meteorological data will be used for meteorological forecasting.

Meteorological forecasting is a vital issue in the field of science around the globe [8,9]. Thus, meteorological forecasting is the application of science and technology to predict atmospheric conditions for a specific location and a given period in the future [10,11,12,13,14,15]. Its purpose is to provide information to relevant individuals and organizations that can be used to reduce losses and improve societal benefits such as property protection, public health and safety, economic prosperity, and quality of life [15]. This forecasting involves data collection, data processing, and data analysis [16,22].

The data flow generated by the meteorological station constitutes time series. A time series is a sequence of observations recorded at regular intervals [17,18,19]. Depending on the frequency of observations, a time series can generally be hourly, daily, weekly, monthly, quarterly, or annual.

But, the decline in the performance of meteorological forecasting models can have a severe impact on both the economy and the environment. As a result, several models have been proposed over the centuries to solve the problem of predictive model performance.

First, numerical weather prediction (NWP) physical models use complex mathematical equations to obtain forecasts based on current weather conditions [22]. These models have long been used to predict events, but they do not allow for short-term predictions [23] and lack accuracy due to the chaotic nature of the atmosphere [23,24]. In addition, the complexity of these models poses significant challenges in their implementation [24]. To reduce the computing power of NWP systems, statistical and machine learning models have been proposed.

Several authors have proposed statistical models for meteorological forecasting. Among these authors, we have, among others, (SHIVAM, et al., 2019) [26], who used ARMA, SARIMA, and ARMAX models to predict temperature and rainfall parameters. (GARIMA and BHAWANA, 2017) [27] used the ARIMA model to predict temperature data time series. However, these methods work in univariate time series and with relatively small data sets [19,21,28]. Similarly, they do not work on nonlinear relationships [19]. Furthermore, they are more limited in predicting a long time horizon or in processing multivariate time series [19,28]. In this case, machine learning tools, in this instance deep learning, would be the ideal tool for time series forecasting according to several authors, such as [18] and [19].

To provide more accurate short- and/or long-term weather forecasts, machine learning techniques can be used to understand and analyze weather patterns. (Azencott, 2019) [29] states that, and I quote: “At the intersection of statistics and computer science, machine learning is concerned with data modeling.” Thus, machine learning for meteorological forecasting uses quantitative meteorological data to build models and attempts to improve model performance by learning from the dataset. According to (Sebastian and Gabriele, 2016) [30], machine learning technologies can provide intelligent models that are much simpler than physical models. It is with this in mind that (Amir et al., 2018) [23] state that ML (machine learning) models are highly accurate compared to physical and statistical models. These machine learning models, in this case offline machine learning models (traditional or conventional machine learning), can numerically formulate the non-linearity of events, based entirely on historical data without the need for knowledge of physical processes [23]. Several offline machine learning methods have been used to develop weather forecasting models, namely LSTM [10,31], CNN [32], and hybrid models (CNN + LSTM) [33]. However, once these meteorological forecasting models are designed using offline machine learning algorithms (LSTM, CNN, LSTM+CNN) and historical data, and then deployed in a real-world environment, their parameters remain static. As a result, these deployed models will encounter new environmental data dynamics that may differ from those used in the historical data. This new dynamic is due to the chaotic nature of the atmosphere, which causes gradual or sudden variations (or changes) in weather conditions. In the context of real-time meteorological forecasting, these characteristics render prediction models based on conventional ML methods (LSTM, CNN, LSTM+CNN) inadequate for accurately modeling environmental dynamics from online data streams, resulting in inaccuracy in predictive models. Similarly, existing conventional ML algorithms require model selection, which is time-consuming and not suited to the context of online learning [34]. Also, in conventional machine learning, these models are trained using the entire dataset at once. This process often requires a lot of computing time and cannot reflect changes in real time. Furthermore, these models based on offline machine learning are neither adaptive nor scalable in real time in the context of real-time meteorological forecasting, because once the models have been trained or formed, their parameters do not change.

In addition, some authors have used new approaches to improve prediction, namely algorithms capable of learning beyond the production phase, which will also enable them to be kept up to date at all times. These approaches are online machine learning methods such as ARIMA Online and LSTCN.

(CHENGHAO et al., 2016) [21] proposed online learning algorithms (ARIMA Online Newton Step and ARIMA Online Gradient Descent) for the efficient estimation of ARIMA model parameters using its recursive formulation in an online learning framework. This online learning approach processes data observations arriving sequentially and updates the models simultaneously. This online ARIMA learning is an online optimization task without noise terms. Finally, in terms of memory cost, their algorithm is independent of the sample size, making it more scalable for processing real-time time series forecasting tasks. Their algorithms have been empirically compared to two online ARMA algorithms. According to their results obtained on synthetic and real data, their proposed algorithms are effective for time series prediction. However, this approach only uses univariate time series. Furthermore, to our knowledge, this technique has not yet been used in the context of meteorological forecasting. There is another online learning approach that, in addition to using univariate time series, also processes multivariate series : the LSTCN approach.

(ALEJANDRO et al., 2022) [20] used long-term and short-term cognitive networks (LSTCN) in their article to predict wind turbine time series in online contexts. These recently introduced neural systems consist of a chain of short-term cognitive network blocks, each processing a block of time data. An LSTCN model is defined as a collection of STCN blocks, each processing a specific time slice (or range) and transferring knowledge to subsequent STCN blocks in the form of weighted matrices. An incoming data block triggers new learning on the last STCN block using the stored knowledge that the network has learned during previous iterations. Then, the prior knowledge matrices are recalculated using an aggregation operator and stored for use as prior knowledge during reasoning. According to their results, the solution of their simulations outperforms traditional neural networks and reports significantly shorter training and testing times. However, to our knowledge, this technique has not yet been used in the context of meteorological forecasting.

The forecasting methods used for the environment, particularly for weather, are based on physical, statistical, or machine learning models [20]. According to [20], although conventional machine learning models often achieve the highest performance compared to other models, their deployment in real-world applications shows inappropriate results in terms of accuracy. This drop in performance is due to the static parameters of these real-time forecasting models because once these conventional ML models are generated, their parameters no longer change, making it impossible to self-adapt or update to the non-linearity of the environment, i.e., to gradual and/or sudden changes in the climate. These variations or changes in meteorology or climate lead to changes in the internal structure of meteorological data, i.e., new distributions of meteorological data. Conventional machine learning is also known as offline machine learning.

One of the main challenges of these models is to improve the accuracy of weather forecasting models. Real-time updates could ensure the accuracy of predictive models in the context of real-time meteorological forecasting. As indicated in [21], enabling forecasting systems to adapt to climate variability and change is a vital necessity.

To overcome these limitations, online learning approaches have been proposed, such as the LSTCN (Long Short-Term Cognitive Network) method [20] and ARIMA-OGD [21]. To our knowledge, these approaches have not been applied in the context of real-time meteorological forecasting. The objective of our study is first to evaluate the performance of these online learning approaches against conventional ML methods (LSTM, CNN, LSTM+CNN), and then to evaluate and compare the performance between these online learning approaches (LSTCN and ARIMA-Online). This evaluation and comparison study will be carried out on univariate and multivariate time series and with short- and long-term horizons.

The remainder of this paper is organized as follows. Section 2 presents the materials and methods. Section 3 illustrates the results and analysis. Section 4 discusses the findings. Finally, Section 5 concludes the paper.

2. Materials and Methods

A time series is a sequence of meteorological observations recorded at regular time intervals [17,18,19]. Depending on the frequency of meteorological observations, a time series can generally be hourly, daily, weekly, monthly, quarterly, and annual [19]. Let x$ϵ$$\mathbb{R}$ be a meteorological variable observed on a discrete time scale over a period t$ϵ$$\{1,2,\dots ,T\}$ where T$ϵ$$\mathbb{N}$ is the number of meteorological observations. Therefore, a univariate time series is defined as a sequence of meteorological observations ${\left\{x^{\left(t\right)}\right\}}_{t=1}^T=\{x^{\left(1\right)},x^{\left(2\right)},...,x^{\left(T\right)}\}$. Similarly, we can define a multivariate series as a sequence ${\left\{X^{\left(t\right)}\right\}}_{t=1}^T=\{X^{\left(1\right)},X^{\left(2\right)},...,X^{\left(T\right)}\}$ of vectors of M weather variables, such as $X^t=\{x_1^{\left(t\right)},x_2^{\left(t\right)},...,x_M^{\left(t\right)}\}$. A model F is used to predict the next time steps in advance, denoted as L, such that $L<T$

We will use two sets of models for our meteorological forecasting simulations of our meteorological data. These are the models based on conventional Machine Learning methods (LSTM, CNN, CNN+LSTM) and the models based on online learning methods (ARIMA-OGD and LSTCN). First of all, we will conduct a performance comparison between these two sets of methods, and then conduct a performance comparison between the online learning models.

These two sets of machine learning models consist of two categories of learning models for meteorological forecasting, mainly SISO (Single-Input Single-Output) and MIMO (Multi-Input Multi-Output) [24]. The SISO is used in univariate time series, while the MIMO is used in multivariate time series. Regarding the SISO model, only one variable is introduced into the learning model to generate only one output. And the MIMO model, on the other hand, allows for the introduction of N variables into the learning model to generate N outputs.

Our simulations will be conducted on univariate and multivariate time series over short-term and long-term horizons. Following the same logic as [24], in our study, the short-term horizon is assigned to 1 hour and the long-term horizon is assigned from 2 hours onward.

Our historical meteorological dataset consists of meteorological data from the city of Ngaoundéré-Cameroon, spanning from $01-01-2011$ to $31-12-2020$, covering a period of 10 years. The frequency of observations is hourly in our dataset. Our dataset contains the following five meteorological parameters: temperature, precipitation, atmospheric pressure, air humidity, dew point. The total number of observations is $87,671$ meteorological observations (or records).

For the simulation of SISO models, we will use only the Temperature parameter. Regarding the simulation of MIMO models, we will use five meteorological parameters.

Subsequently, our dataset will be divided into $80\%$ for training data and $20\%$ for test data. The metrics used will be MAE, training time, and testing time.

We will conduct an evaluation and performance study between the models to determine the most performing model in the case of univariate series on one hand and in the case of multivariate series on the other hand. this study will be conducted based on short-term and long-term horizons.

The programming language used to code all these methods is Python, version $2.9.1$. Our simulations will be performed on the Jupiter Notebook IDE. Our PC on which our simulations will be performed has the following characteristics: Our PC on which our simulations will be performed has the following characteristics: Brand HP, operating system (Windows 10 Professional 64-bit), memory $\left(4GB\right)$, hard drive $\left(500GB\right)$, processor type (AMD), and processor speed $\left(1.80GHz\right)$.

3. Results and Analysis

The results and our analyzes will be done respectively on univariate time series (for SISO models) and multivariate time series (for MIMO models). Let’s start with the results and our analyzes on univariate time series.

It should be noted that our results on univariate time series are based solely on the temperature parameter. Table 1 presents the metrics of our different models in the case of univariate series for a prediction horizon of 1 hour.

Table 1 shows that the LSTCN model performs best in terms of mean absolute error ($\mathbf{5020}\ast {\mathbf{10}}^{-\mathbf{6}}$). It also has the best execution time for test data ($\mathbf{0}.\mathbf{053}$ seconds). In addition, it is considered the second-best model in terms of training time ($\mathbf{20}.\mathbf{28}$ seconds). In contrast, the ARIMA OGD model performs the worst in terms of mean absolute error ($\mathbf{7828}\ast {\mathbf{10}}^{-\mathbf{6}}$). However, it has the fastest training time ($\mathbf{5}.\mathbf{85}$ seconds).

Table 2 and Table 3 present the metrics of our different models for long-term horizons, namely: a 2-hour horizon for Table 2 and a 12-hour horizon for Table 3.

Regarding Table 2 and Table 3, for the long-term horizon, the LSTCN model performs best in terms of mean absolute error, with $\mathbf{11499}\ast {\mathbf{10}}^{-\mathbf{6}}$ for the 2-hour prediction horizon and $\mathbf{33464}\ast {\mathbf{10}}^{-\mathbf{6}}$ for the 12-hour prediction horizon. It also has the shortest testing time for both prediction horizons, namely 2 hours ($\mathbf{0}.\mathbf{06}$ seconds) and 12 hours ($\mathbf{0}.\mathbf{33}$ seconds). In addition, this model is considered to be the second best model with a reduced training time of $\mathbf{8}.\mathbf{3}$ seconds. Furthermore, the ARIMA OGD model is the least performant model in terms of mean absolute error in the long-term horizons. However, this model is considered to be the best model in terms of training time for the 2-hour ($\mathbf{5}.\mathbf{15}$ seconds) and 12-hour ($\mathbf{5}.\mathbf{12}$ seconds) prediction horizons. After analyzing our various SISO model results, we will now analyze the various MIMO model results.

With regard to the tables below, Table 4 presents the evaluation metrics for our different models for a short-term prediction horizon (1 hour), while Table 5 and Table 6 illustrate the evaluation metrics for our different models for long-term prediction horizons, namely a 2-hour prediction horizon for Table 5 and a 12-hour prediction horizon for Table 6. Furthermore, in the tables below, the ARIMA OGD model is excluded because this model does not take multivariate series into account.

Regarding Table 4 presenting the metrics for the short-term horizon, Table 5 and Table 6 for the long-term horizon for MIMO models, the LSTCN model is the most performing model in terms of MAE and in terms of training and testing time.

4. Discussion

Our simulations show that online learning models have a faster training and testing time than conventional Machine Learning models on univariate and multivariate time series.

Although previous studies have not conducted a comparative analysis between the LSTCN and ARIMA OGD models, and performed simulations on meteorological data, we conducted our study in this direction. According to our results, the forecast data generated by the LSTCN model is more accurate than the ARIMA OGD model on univariate time series for both short-term and long-term forecasts. Moreover, the training time of the LSTCN model is slightly longer than that of the ARIMA OGD model. The ARIMA OGD model is only restricted to univariate time series, which limits the simulation on multivariate time series. And yet, the LSTCN extends the simulation to multivariate time series.

The LSTCN model is also the most performant model compared to conventional Machine Learning models, whether in terms of MAE, training time, or testing on univariate and multivariate time series for short-term and long-term forecasts.

Regarding long-term predictions for univariate and multivariate series, it turns out that the accuracy of forecast data can degrade as the prediction horizon increases.

Based on the results of our simulations, we were able to observe that LSTCN model are better suited for real-time weather forecasting.

5. Conclusions

In summary, weather forecasting is the application of science and technology to predict weather conditions for a specific location and the given period in the future. Thus, several forecasting methods used for the environment, particularly for weather, are based on physical, statistical, or machine learning models. although conventional machine learning models often achieve the highest performance compared to other models, their deployment in a dynamic environment shows a certain inadequacy, namely the chaotic nature of the atmosphere in the context of real-time weather forecasting. Updating conventional models with new meteorological information is often very costly. This update could ensure the accuracy of the predictive model. To address these limitations, online learning approaches have been proposed, such as the LSTCN method and ARIMA-OGD. These approaches were compared against conventional Machine Learning methods, namely LSTM, CNN, and CNN+LSTM. According to the results, the LSTCN model is the most effective compared to conventional machine learning methods in terms of training time, testing time, and accuracy on short- or long-term univariate or multivariate time series. Similarly, this model outperforms the ARIMA OGD (Online Gradient Descent) model in terms of accuracy and testing time. However, its training time is slightly longer than that of the ARIMA OGD model. LSTCN model is better suited for real-time weather forecasting.