A Novel Method of Forecasting Chaotic and Random Wind Speed Regimes Based on Machine Learning with the Evolution and Prediction of Volterra Kernels

1. Introduction

Wind energy is one of the main sources of the recent increase of renewable energy systems worldwide. Its potential output depends greatly on wind characteristics such as speed, which requires accurate forecasting to manage and control grid networks and standalone electrical power operation [1]. Wind speed prediction is normally analyzed by physical models or statistical models. Physics-dependent models are based on metrological physical laws, such as wind speed and direction [2], whereas statistical analysis relies on data algorithms such as moving average, autoregressive, machine learning, support vector machine [3,4], and hybrid methods combining different machine learning algorithms [4,5]. Statistical modeling is useful for short-time forecasting which ranges from hours to even seconds, whereas physics-driven models are more suitable for long-time prediction of wind speed.

The chaotic and random nature of wind speed is one of the shortcomings effects of wind forecasting due to the indeterminate nature of the wind speed. This problem is exploited even more with a low-scale wind speed regime. Prediction using the nonlinear learning ensemble method [5] is attempted to investigate deep learning time series, whereas a frequency domain analysis based on frequency decomposition [6] is conducted. Recently, a probabilistic wind speed prediction model was attempted using real-time decomposition [3]. However, dealing with chaotic and random forecasting [7] requires other approaches, such as fractal analysis [8,9], which calculates the persistence of past features of monotonous nature. It has been introduced to characterize and quantify the complexity of geometries with a fractal factor ranging from 1 to 2. Another approach is wavelet transformation analysis which converts a series of data from the time domain to several frequency filtrations. Earlier, the largest Lyapunov exponent method [10] was used for continuous and discrete nonlinear systems to measure variations in the Lyapunov exponent trajectories. A similar method in this context is the Kolmogorov entropy method [11].

The chaotic and random characteristics of wind speed can better be described using the Volterra series due to the nonlinearities relating to inputs and outputs. The calculation of Volterra series and kernels suffers from poor accuracy and large complexity, hence conventional and time-delay series neural networks [12,13,14] are used both in time and frequency domains to overcome this shortcoming, by truncating and recurrent estimation approaches [15,16]. Yet, the selection of NN layers, activation functions, and input and hidden layers nodes, impose difficulties for the optimal analysis of estimating the Volterra kernels, which describe the input-output of the nonlinear system. With a smaller number of neurons, lower accuracy is achieved, whereas, with a larger neuron number, overfitting occurs. In the case of a time-delay artificial neural network (TDANN), the length of the memory affects accuracy too, because with shallow memory length, the lower correlation between neurons has resulted, and with deep memory length, overfitting is obtained [17]. Wavelet decomposition was used to analyze and predict wind speed of chaotic nature [18]. The work of [19] is used to simplify NN training by truncating the Volterra kernels. Normally, up to the first third-order time-domain kernels are needed for reasonable accuracy, but the tradeoff is the size of the input range of values.

In this work, a novel method is proposed and implemented to forecast random, chaotic wind speed using different hourly and daily data sets, for training ANN to evaluate the first third-order Volterra kernels over a period of several months. The adoption of data sets depends on the randomness and variability of the measured data. Then, the behavior of the kernels is predicted by finding their trend values from their previous evaluations. Due to the complexities of estimating many kernels, only the first-order and second-order kernel coefficients are calculated. Hence, with two layers, a 4-input neutron, and 4-hidden neutrons, the network is implemented. The method has been compared with two other methods of evaluating the forecasts [20] using the same data sets.

2. Methodology

2.1. Wind Speed Dataset

Wind speed has been measured over a period of 12 months by a logger located 10 m above ground at Fujairah site in the UAE, with site data as shown in Table 1. The dataset is logged as one-minute average values. The wind regime is normally chaotic and random with large indeterminate periods. Figure 1 depicts the nature of the wind regime at the site [21]. It can be deduced that wind speed is generally calm with the maximum values scarcely reaching 10 mph. This can further impose difficulties in predicting future wind speeds. Selections that make a careful selection of the dataset are crucial to eliminating random errors.

The characteristics of Fujairah site is listed in Table 1. The wind regime at site is largely random in nature with large variations from season to season. It must be mentioned here that during the 2020-2021 measurements, the wind speed was mainly calm with an average speed of 5 mph. The logger setup is basic, which is meant for academic research purposes. The metrological data including wind speed, are recorded in two ways, wirelessly and wired to a LabVIEW^© programmed platform.

Due to the nature of large wind randomness, it is important to select periods of random wind speed persistence. Consequently, it is intended to use the dataset for both hourly and daily prediction with different selections, for short-time and long-time forecasting cases. For the hourly forecasting, only 100 appropriate days are selected for the analysis of this study due to being exposed to noticeable seasonal variations. The exposed days are divided equally into ten periods of 10 days each. The wind speed is selected 4 times a day, covering an average of 6 hours each in which normal potential variations can exist, making it suitable for the goal of this study. For the daily forecasting, wind regime exploitation of the entire 12 months period is used with daily average values for each month.

Another difficulty of accurate forecasting is the size and amount of calculating each Volterra kernel for the selected period. For instance, an N perception neuron network requires $m(N^2+N+1)$ calculations for up to the m-order kernels, and for each period that is considered for this study. The prediction of these kernels requires neural networks for each kernel to be determined. In this work, 2-layers NN with only 4 neutrons is chosen, which makes network training inaccurate and difficult.

2.2. NN Model

It is known that dynamic neural networks, which may have topologies of recurrent and time-delayed patterns, can identify, and solve nonlinear dynamic systems. In the first stage of the analysis, a conventional artificial neuro network (ANN) is implemented for the extractions of the Volterra kernels, with one hidden layer to accommodate a smaller number of Volterra kernels. The inputs are the time-delayed data measured with an M-depth of history according to the dataset selected and explained in the previous section. The number of neurons used in the input layer constitutes the memory depth of the data used. It’s also appropriate to use an equal number of neurons in the hidden layer as well for the convenience of Volterra kernel calculations. A general M-memory depth CNN is depicted in Figure 2, with p neurons in the hidden layer, and one neuron output. In this study, only 4 specific periods of interest are chosen as inputs that reflect the persistence of the wind speed randomness, incorporated with 4 neurons in a single hidden layer.

The next stage of analysis is to estimate the evolution of the Volterra kernels throughout the different selected periods of the dataset. This has been attempted using either statistical analysis or machine learning. In the latter, a similar topographic pattern of Korhonen’s self-organizing feature map (SOFM) network is proposed for the 4 × 4 and 1 × 4 neurons.

The Volterra kernels are functions of layer-to-layer weights and neuron biases [17]. Hence, it’s needed first to train the neural network with the datasets explained above to evaluate all weights and biases. Then Volterra kernels up to the third order are calculated for each period studied. Afterward, the evolution of the main Volterra kernels values is predicted and used to forecast future wind speeds.

2.3. Procedural Algorithm

Figure 3 depicts the procedural algorithm adopted in this study, which relies on the Volterra kernel evolution from one period to the next. In each period, up to the third-order kernels are evaluated using static artificial neural networks with time-delayed wind speeds that have been recorded in the datasets. In the process of the kernel evaluation, the weight coefficients of the input-hidden layer, optimal weights of the output layer, as well as biases of both the hidden layer and output layer, are extracted from the trained networks. Then, the mathematical Volterra analysis is applied according to the formulation displayed in the next section.

2.4. Volterra Kernel Extraction

With Volterra modeling, a single output is determined from a series of inputs using mathematical formulation without necessarily the need of knowing the real structure of the nonlinear system, i.e.,

$$y\left(k\right)=F\left(u\left(k\right)\right)$$

(1)

That’s, given a set of m-memorized inputs $U\left(k\right)=\left\{u\left(k\right), u\left(k-1\right), \mathrm{..}, u\left(k-m+1\right)\right\},$the output $y\left(k\right) =u\left(k+1\right)$can be evaluated in discrete form as follows,

$$\begin{array}{c}y\left(k\right)=b_0+{\displaystyle \sum }_{j=1}^p{w}_j\phi \left(b_j\right)+{\displaystyle \sum }_{j=1}^p{w}_j {\phi }^{\left(1\right)}(b_j) \left[u\left(k\right)+w_{2,j} u\left(k-1\right)+\mathrm{..}+w_{M+1,j} u\left(k-M\right)\right]\\ +{\displaystyle \sum }_{j=1}^p{w}_j\frac{{\phi }^{\left(2\right)}(b_j)}{2!}[u\left(k\right)+w_{2,j}u\left(k-1\right)+\mathrm{..}+w_{M+1,j}u\left(k-M\right){]}^2\\ +{\displaystyle \sum }_{j=1}^p{w}_j\frac{{\phi }^{\left(3\right)}(b_j)}{3!}[u\left(k\right)+w_{2,j}u\left(k-1\right)+\mathrm{..}+w_{M+1,j}u\left(k-M\right){]}^3\end{array}$$

(2)

This can be simplified using Volterra kernels as,

$$y\left(k\right)= h_0+ {\displaystyle \sum }_{m_1=0}^M{h}_1 ( m_1) u\left(k-m_1\right)+ {\displaystyle \sum }_{m_1}^M{\displaystyle \sum }_{m_2}^p{h}_2(m_1,m_2) u\left(k-m_1\right)u\left(k-m_2\right)$$

(3)

where $h_p(m_0$,$m_1, \mathrm{..}{m}_p)$ is the p-order Volterra kernels which constitutes weighting coefficients on past values of m-memorized inputs. In this context, only kernels of p value up to 2 are determined. The first up to the second-order kernels are defined as

$$h_0=b_0+{\displaystyle \sum }_{j=1}^p{w}_j \phi \left(b_j\right)$$

(4)

$$h_1\left(m\right)={\displaystyle \sum }_{j=1}^p{w}_j w_{m+1, j} {\phi }^{\left(1\right)}\left(b_j\right), m=0,1,\mathrm{..},M$$

(5)

$$h_2\left(m_1,m_2\right)={\displaystyle \sum }_{j=1}^p{w}_j w_{m_1+1,j} w_{m_2+1,j} {\phi }^{\left(2\right)}\left(b_j\right), m_1=1,2,\mathrm{..}M, m_2=1,2,\mathrm{..},M$$

(6)

In this work, the Volterra kernels are first evaluated using the weight coefficients of the implemented neural network from (4, 5, 6) for the period in concern, hence the forecasted wind speed is evaluated from (3). It’s accepted to truncate these kernels up to the 3-rd order, to reduce computational efforts of this work without raising extra errors.

2.5. Kernels Estimation

2.5.1. Prediction methods

Predicted Volterra kernels are estimated using previous kernels extractions in the last 10 months period. In this work, only a single value of h_0, a vector of 4 values for h₁, and a two-dimensional area of 4 × 4 values of h₂ are used for their prediction using statistical analysis as well as machine learning.

2.5.2. ML

Once Volterra kernels are extracted from previous periods, different sizes of neural networks are further used to predict evaluated values of the h₀, h₁ and h₂ kernels for future estimation. Figure 4, Figure 5 and Figure 6 display the neuro networks which are implemented to extract h₀, {1 × 4} h₁ and {4 × 4} h₂ respectively.

It is noted that Korhonen neural network is important for the evaluation of the zero and first order Volterra kernels. Due to large computations, the Korhonen neural arrangement has not been applied for evaluating the second order kernels. This truncation of Volterra kernels can cause enormous errors whereas the use of smaller number of neurons per NN-layer will reduce the accuracy of this analysis.

3.1. Volterra Kernels Evaluation

3.1.1. Long-term Prediction

Table 2 lists the zero- and first-order evaluated kernels in the 12 months period. The forecasting of wind speed is performed based on only these two kernels as an accepted approximation to reduce exhausted computations, but this would impose further errors on the proposed forecast.

It is deduced from above table that based on previous months’ data, the forecasted wind speed can be expressed truncated as shown in (3) to be $2.271605-0.60568* S1-1.31348* S2+1.223976*S3$, where S₁, S₂, S₃ are the measured wind speed on 3 previous days. It is intended here to check the amount of daily wind speed randomness, which is expected to be large, and consequently the estimation is based merely on statistical analysis.

3.1.2. Short-term Prediction

As a first attempt, a period span of 100 days is considered for the short-term kernels prediction, The dataset is divided into two periods of 50-day each to monitor the variety and extend of speed randomness. Four selected daily measurements which reflect potential wind speed randomness, are chosen in the analysis. The resultant zero- and first-order Volterra kernels are:

$$h0$$	$$-2.5968$$
$$h1$$	$$\left\{2.0140 0.1821 0.0510 0.3577\right\}$$

Using only these values of kernel coefficients, Figure 7 shows both the measured and forecasted wind speed for the 100-day period, divided into two 50-day periods,

The proposed algorithm of extracting the kernels and estimating their evaluated values, has been applied on this period. In Table 3 the two-order Volterra kernels are listed for 10 consequence periods of 10 days each, based on the occurrence of potential persistence of wind speed randomness. The Volterra kernels up to the second-order coefficients are calculated for specific hours of the days, normally during 1 AM-8 AM, in each day of logged data. These specific hours constitute the largely persistence of wind speed randomness. Other time occurrences can also be considered.

hence, the forecasted wind speed is determined by,

$$Speed=3.4194+4.5363* S1-0.48691* S2-2.9585*S3+1.306*S4,$$

when neural networks are implemented, and as,

$$Speed=0.1352-0.0932*S1-0.0166*S2-0.3929*S3+0.48*S4,$$

when statistical analysis is applied, where S₁, S₂, S₃, S₄ are the measured wind speed on 4 previous days.

3.2. Wind Speed Forecasting

Wind speed forecasting is performed in two different phases in the first, wind speed is calculated with the extracted Volterra kernels up to the third order and compared with the measured values. Figure 8 displays the two patterns of measured and calculated wind speed for each of the 10 periods of 10 days each. It can be noticed that although the two concurrent patterns are similar, the errors are enormous. This is due to three main reasons:

• The overall analysis is based on the evaluation of up to the third order Volterra kernels.
• Only 2-layers neural networks of 4 neutrons per layer are incorporated which implies that machine training was not perfect.
• The forecasted wind speed in each period has been compared with the measured one in a handicapped manner since evaluated Volterra kernels are for the entire period, but not for each day as has been performed.
• The historical depth of memory recurrence of selected inputs was limited to 4 only. The reason for this is to compensate for the large number of needed computations.

3.3. Method Validation

The selection of proper dataset is crucial for validating the proposed algorithm in this study. It was intended to compare measured wind speed with the forecasted value based on statistical trends. It was unable to validate the implemented method, since there were large discrepancies between the measured and forecasted values, as depicted in Table 4. For the first dataset, Volterra kernels were calculated in a 12-month period, and the resultant is extracted from the evolution of previous 12 months. Three main readings per day of the 12th month, which reflect potential variations in wind speed, were used in this analysis.

In Table 5, the dataset period of 100 days divided into 10 groups, that reflects major variation in the wind speed is selected, in which 4 major readings per day are used for evaluating the resultant Volterra kernels. With this arrangement, a better estimation of the wind speed forecast is obtained. Two ways of forecasting the wind speed are applied, one from statistical trends, and the other from machine learning, in which a neural network is used for each kernel coefficient. In this context, only zero-order and first-order kernels of 4 coefficients are considered.

4. Conclusions

A novel method is used to evaluate the Volterra coefficients up to the third-order kernels, for forecasting wind speeds of chaotic nature with variable and inconsistent randomness. The Volterra kernels are extracted for a sequence of periods to estimate predicted evolution and hence forecasted wind speed. Computational efforts are traded for the complexity of Volterra mathematical analysis. Errors arise due to the following:

• The overall analysis is based on the evaluation of up to the third order Volterra kernels.
• Only 2-layers neural networks of 4 neutrons per layer are incorporated which implies that machine training was not perfect.
• The forecasted wind speed in each period has been compared with the measured one in a handicapped manner since evaluated Volterra kernels are for the entire period, but not for each day as has been performed.
• The historical depth of memory recurrence of selected inputs was limited to 4 only. The reason for this is to compensate for the large number of needed computations.

Nevertheless, as the aim of this work is to address forecasting nonlinearities with Volterra kernels, this is an incentive attempt for other collaborators to opt for more accurate results, by eliminating the above-mentioned errors.