A Multi-scale Hybrid Wind Power Prediction Model Based on LSSVR-RELM-Multi-Head Attention-BiGRU and Data Decomposition

1. Introduction

Wind energy is an environmentally friendly and economically viable form of renewable energy. According to the Global Wind Energy Council (GWEC)’s Global Wind Report 2023, it is projected that global onshore wind power capacity will exceed 100GW for the first time by 2024 [1]. However, wind energy data exhibits significant randomness and non-stationarity, which has a substantial impact on the stable operation of power systems. Therefore, accurate prediction of wind power is crucial [2,3,4].

Currently, wind energy forecasting methods are mainly divided into physical models, statistical models, and artificial intelligence models [5]. Physical models primarily consider various geographical information, essentially based on Numerical Weather Prediction (NWP) [6] and high-precision wind farm simulation strategies. This approach has been widely applied in countries such as Spain, Denmark, and Germany, with examples including the SOWIE model developed by Eurowind in Germany [7] and a wind sequence correction algorithm based on NWP proposed by Wang et al. [8]. These algorithms utilize large amounts of data to calculate accurate and reliable wind power predictions. However, due to the large data scale, physical models suffer from slow computational speed and low efficiency, and are affected by adverse wind farm conditions, making reliable data collection difficult. Statistical methods, on the other hand, do not consider external conditions such as geography or electrical factors. Their core is to use the relationships between historical wind power data for prediction in order to improve prediction efficiency [9]. Classical models such as Moving Average (MA) [10] and Autoregressive Integrated Moving Average (ARIMA) [11] are based on modeling linear relationships between data. However, when facing complex patterns, these methods suffer from issues of low prediction accuracy and poor performance.

In comparison to the aforementioned approaches, artificial intelligence models have exhibited increasingly remarkable performance in the domain of wind energy prediction. Methods such as Support Vector Regression (SVR) [12], Extreme Learning Machine (ELM) [13], and Gated Recurrent Unit (GRU) [14] have yielded significant accomplishments in wind energy prediction research. However, due to the considerable prediction errors commonly associated with individual models, hybrid prediction models have gained widespread adoption in recent years. Presently, hybrid models primarily encompass three facets: data preprocessing, optimization algorithm tuning, and the prediction of single or combined models. Zhang et al. proposed a method employing Wavelet Transform (WT) for data denoising preprocessing and enhanced the Improved Atomic Search Optimization (IASO) algorithm to optimize the parameters of Regularized Extreme Learning Machine (RELM) [15]. The accuracy of this model was substantiated through ablation experiments. Wang et al. optimized the input weights of ELM using genetic algorithms [16], whereas Zhai, Ma, and Tan utilized the Artificial Fish Swarm Algorithm and Salp Swarm Algorithm to optimize the initial input weights and thresholds of ELM [17,18]. The outcomes indicate that these models manifest high prediction accuracy. In recent years, the wind power prediction domain has begun embracing deep learning models such as Long Short-Term Memory (LSTM) [19,20], Temporal Convolutional Neural Network (TCN) [21], and Bidirectional Gated Recurrent Unit (BiGRU) [22]. These models and their derivatives have emerged as principal tools in this sphere. Scholars like W. Wang proposed a prediction methodology based on the fusion of TCN and Light Gradient Boosting Machine (LightGBM) [23]. Researchers such as Chi integrated the attention mechanism into the BiGRU-TCN hybrid model and employed wavelet denoising (WT) processed raw data for prediction[24]. Experimental results corroborate the robust predictive capability of this model. Presently, researchers generally favor hybrid models grounded in intelligent algorithms. Ding et al. advanced a wind energy prediction model optimizing ELM using the Whale Optimization Algorithm (WOA) [25]. Ye et al. preprocessed raw data using clustering algorithms and optimized ELM parameters using Genetic Algorithm (GA) to realize short-term wind speed prediction [26]. Zhang et al. proposed a sparse search algorithm (SSA) to optimize the TCN-BiGRU model and employed the Variational Mode Decomposition (VMD) algorithm to decompose data, thereby mitigating the non-stationarity of wind power data [27]. Ablation experiments demonstrated that this model achieved heightened prediction accuracy compared to scenarios where the SSA algorithm was not employed for parameter optimization. The research by the aforementioned scholars underscores that hybrid models, predicated on algorithmic parameter optimization, can further enhance model prediction accuracy.

Wind power data is inherently characterized by randomness and non-stationarity, necessitating data preprocessing to effectively mitigate prediction errors. To address this, scholars have proposed methodologies grounded in signal decomposition for model formulation. This approach principally leverages signal decomposition algorithms to partition initial wind energy data into multiple regular sub-modes, enabling independent prediction of each sub-mode. Empirical evidence has underscored the efficacy of this approach in substantially reducing prediction errors. For instance, Gao et al. [28] introduced a composite model combining Empirical Mode Decomposition (EMD) with GRU for prediction tasks. However, the EMD algorithm encounters notable challenges such as mode mixing when confronted with gapped signals. In response, scholars have advocated for the incorporation of uniformly distributed white noise into EMD, manifesting as Ensemble Empirical Mode Decomposition (EEMD) and Complementary Ensemble Empirical Mode Decomposition (CEEMD). This technique has found widespread adoption within the prediction domain. For instance, Wang et al. [29] utilized the EEMD method to decompose photovoltaic generation data into high-frequency, mid-frequency, and low-frequency components, subsequently predicting each component individually and aggregating the predictions to yield the final forecast. Torres et al.[30] argued that inadequate decomposition processing frequencies may lead to the persistence of white noise's influence and the emergence of pseudo-mode phenomena. Consequently, they proposed the Complementary Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), effectively mitigating residual white noise's impact. Addressing residual noise and pseudo-mode concerns further, Colominas et al.[31] introduced the Improved Complementary Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN), demonstrating heightened reconstruction accuracy for components and enhanced suitability for nonlinear signal analysis. For instance, Mohaned L B et al. [32] proposed a tool wear detection method based on spectral decomposition and ICEEMDAN mode energy, with experimental findings showcasing enhanced detection precision using this decomposition method.

In summary, several limitations persist within the realm of wind power prediction. Firstly, prevalent approaches tend to employ single-scale prediction models for all decomposed sub-modes, overlooking the distinctive characteristics of sub-modes across varying frequencies. Secondly, contemporary artificial intelligence prediction methods often encounter challenges in parameter optimization, resulting in high trial and error costs. Lastly, some conventional optimization algorithms exhibit insufficient optimization capabilities and sluggish convergence speeds. Therefore, this paper advocates for a multi-faceted approach, merging data decomposition techniques with multiple models, and harnessing the Differential Bees Optimization algorithm (DBO) [33]. This strategy rectifies existing wind power model shortcomings by addressing deficiencies in sub-mode prediction methods, parameter optimization, and scale singularity. Specifically, ICEEMDAN is employed to decompose the original wind power sequence into modes, yielding multiple sub-modes. Subsequently, all sub-sequences are reconstructed into several new components using Permutation Entropy (PE), with high, mid, and low-frequency components determined based on PE values. Following this, LSSVR, RELM, and MHA-BiGRU models are established to predict high, mid, and low-frequency components respectively. The DBO algorithm is then utilized to optimize the MHA-BiGRU model. Finally, the predicted values of each frequency are aggregated to obtain the final prediction result.

The main contributions of this paper are as follows:

(1) Proposing a multiscale wind power prediction hybrid model combining ICEEMDAN, PE, LSSVR, RELM, and MHA-BiGRU. Utilizing ICEEMDAN to decompose the original wind energy sequence. Compared with methods such as EMD, EEMD, and CEEMDAN, this method can more effectively address mode mixing and residual noise problems, thereby better handling the nonlinearity and non-stationarity characteristics of wind power sequences.

(2) Utilizing the strong noise resistance and low computational complexity advantages of PE to calculate the entropy values corresponding to the complexity of each sub-mode, and reconstructing the sub-components in phase space according to the size of the entropy values. By reconstructing into high, medium, and low-frequency components, further eliminating the non-stationarity characteristics of wind power data and optimizing the input data of the model.

Figure 1. Flowchart of the proposed ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model.

Figure 1. Flowchart of the proposed ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model.

(3) By utilizing the characteristics of different models in different frequency predictions, the high, medium, and low-frequency components obtained through decomposition are inputted into LSSVR, RELM, and MHA-BiGRU models respectively for prediction, thereby achieving targeted multiscale hybrid prediction and overcoming the problem of insufficient prediction accuracy of a single model.

(4) Utilizing the powerful capability of multi-head attention mechanism to capture the strong correlation between data, combined with the BiGRU model, can comprehensively and accurately capture the information in the sequence, solve the problem of information overload when predicting with the BiGRU model due to the inability to perform parallel computation, and thus improve the accuracy of model prediction. Then, predicting the low-frequency sequence, in order to address the limitations and blindness of manual parameter tuning in the MHA-BiGRU model when there are too many parameters, optimizing the learning rate, the number of BiGRU neurons and multi-head attention heads, as well as the number of filters and regularization parameters through the DBO algorithm to further improve its prediction accuracy.

(5) Conducting multiple groups of ablation experiments on the proposed model,verifying the superiority of the proposed model from the perspectives of evaluation indicators, error analysis, and linear fitting. The model is compared with 8 benchmark models and validated using data sets from 4 different seasons. The results show that the proposed model improves the accuracy of wind power prediction and achieves better prediction results.

2. Methods

2.1. Intrinsic Combined Ensemble Empirical Mode Decomposition with Adaptive Noise

Empirical mode decomposition (EMD), proposed by Huang et al. in 1998, is a signal processing method suitable for nonlinear and non-stationary processes. However, after decomposing data using this method, there is a significant issue of mode mixing. Subsequently, Wu et al. proposed the Ensemble Empirical Mode Decomposition (EEMD) method to address this issue. EEMD introduces white noise with a mean of 0 into the decomposition process, uniformly distributing it across the entire time-frequency space to mask the inherent noise of the signal, thereby alleviating the problem of mode mixing caused by EMD decomposition. Nevertheless, when the number of decomposition iterations in EEMD is insufficient, the presence of white noise may lead to reconstruction errors.

To overcome the limitations of EMD and EEMD, Torres et al. proposed the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) method. CEEMDAN replaces the uniformly distributed white noise signal with IMF components containing auxiliary noise obtained after EMD decomposition, thereby reducing the influence of residual white noise components.

However, CEEMDAN still cannot completely address the issues of residual noise and pseudo modes. Therefore, Colominas et al. further improved CEEMDAN by introducing the Iterative Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) method. The core of this signal decomposition method lies in selecting the Kth IMF component of the white noise decomposed by EMD as the auxiliary noise. Through multiple iterations of noise addition and decomposition, ICEEMDAN comprehensively addresses the randomness and non-stationarity of the data, thereby enhancing the stability and reliability of the final decomposition results and reducing residual noise generated during reconstruction. The computational process is as follows:

1. Based on the original signal s, construct a new sequence  $s^i$  by adding i groups of white noise  ${\omega }^i$  to s, resulting in the first group of residues  $R_1$ .

$$s^i=s+{\alpha }_0{E}_1({\omega }^i)\text{，}{R}_1=\langle M(s^i)\rangle$$

(1)

where  $E_k(\cdot )$ represents the k-th mode component generated by EMD decomposition,  $M(\cdot )$  represents the local mean of the signal generated by the EMD algorithm, and  $\langle \cdot \rangle$  represents the overall mean.

2. Calculate the first mode component $I_{\mathrm{IMF}1}=s-R_1$ iteratively to obtain the k-th group of residues $R_k$ and mode component $I_{\mathrm{IMFk}}$ .

$$R_k=\langle M(R_{k-1}+{\alpha }_{k-1}{E}_k({\omega }^i))\rangle$$

(2)

$$I_{\mathrm{IMFk}}=R_{k-1}-R_k$$

(3)

where ${\alpha }_k$ can be expressed as:  ${\alpha }_k=\{\begin{array}{l}{\epsilon }_0\mathrm{std}(\lambda )/\mathrm{std}(E_1({\omega }^i)),\lambda =s\\ {\epsilon }_0\mathrm{std}(\lambda ),\lambda =R_k,k=1,2,\cdots ,K\end{array}$ 3. Repeat step 2 until the calculation is complete, obtaining all mode components and the final residue.

2.2. Permutation Entroy

The PE (Permutation Entropy) algorithm, introduced by Bandt et al., serves as a method for characterizing the complexity of time series. Its core principle lies in assessing the irregularity of a time series through the examination of permutation patterns within its subsequences. A higher entropy value signifies greater complexity within the time series, whereas a lower value indicates a higher degree of regularity. The computational formula for this algorithm is as follows:

1. Consider a time series  $X=\left\{x(i),i=1,2,\cdots ,k\right\}$ .

2. Perform phase space reconstruction on the time series, resulting in a reconstruction matrix Z with a given dimension m and time delay  $\tau$ .

$$Z^T=\left[z(1)z(2)\cdot \cdot \cdot Z(k-(m-1)\tau )\right]$$

(4)

where $z(j)=\left\{x(j),x(j+\tau ),\cdots ,x(j+(m-1)\tau )\right\}$

3. Sort the elements of  $z(j)$  in ascending order and record the sequence of elements in each row of the reconstruction matrix. Calculate the probability of occurrence for each element sequence to obtain  $p_1,p_2,\cdots p_q$ .

4. The permutation entropy of time series X is defined as:

$$H\left(m\right)=-{\displaystyle {\sum }_{k=1}^q{P}_k\ln P_k}$$

(5)

where  $P_k$  represents the probability of each element size relationship permutation in the reconstruction matrix Z, m is the given dimension, k is the number of subsequences, and q is the total number of elements.

2.3. Bidirectional Gated Recurrent Unit

The GRU (Gated Recurrent Unit) stands as an enhanced version of the Long Short-Term Memory (LSTM) network within the domain of recurrent neural networks. It is tailored to capture long-term dependencies within sequential data while boasting fewer parameters than LSTM, thereby mitigating computational costs. The core principle involves amalgamating the forget gate and input gate into a unified update gate. Through the management of information flow and state updates, it effectively reduces the parameter count and computational overhead. The model's architectural depiction is presented in Figure 2, accompanied by the following computational formulas:

$$z_t=\sigma \left(W_z\cdot \left[h_{t-1},x_t\right]\right)$$

(6)

$$r_t=\sigma \left(W_r\cdot \left[h_{t-1},x_t\right]\right)$$

(7)

$${\widehat{\mathrm{h}}}_t=\tanh \left(\mathrm{W}\cdot \left[r_t\odot {\mathrm{h}}_{t-1},x_t\right]\right)$$

(8)

$${\mathrm{h}}_t=\left(1-z_t\right)\odot {\eta }_{t-1}+z_k\odot {\widehat{\mathrm{h}}}_t$$

(9)

where $z_t$ represents the update gate,  $r_t$ ,  $x_t$ ,  ${\mathrm{h}}_t$ ,  ${\widehat{\mathrm{h}}}_t$  represent the reset gate, current time step input information, new cell vector, and hidden state vector, respectively.  ${\mathrm{W}}_z$ ,  ${\mathrm{W}}_r$ , W represent the weight matrices.

Due to the strong temporal characteristics inherent in wind power load data, information corresponding to both the previous time step (t-1) and the current time step (t+1) significantly impact the prediction results at time t during model training. Consequently, the GRU model fails to fully exploit the inherent information within wind power sequences. In contrast, the Bidirectional Recurrent Neural Network (BiGRU) model addresses this limitation by utilizing both past and future data to enhance prediction accuracy. This effectively overcomes the drawback of low data information utilization observed in the GRU network. Comprised of two GRU models, BiGRU possesses the capability to capture bidirectional dependencies within sequential data, thereby enabling it to adapt to more complex sequence patterns. The network structure is visually depicted in Figure 3.

2.4. Multi-Head Attention

The Attention Mechanism (AM) is a computational method for efficient allocation of information resources, prioritizing more crucial tasks to effectively mitigate information overload.In AM, input information is represented by key vectors (Keys) and value vectors (Values), while target information is represented by query vectors (Query).The weights of the value vectors are determined based on the similarity between the query vector and the key vector. Following this, the final attention values are computed by aggregating the weighted value vectors. The fundamental formula is as follows:

$$S_{att}=W\times V$$

(10)

$$W=func\left(Q,K\right)$$

(11)

where $S_{att}$ is the attention value, Q represents the query vector, K represents the key vector in the key-value pairs, V represents the value vector in the key-value pairs, W represents the weight corresponding to V, and $func(\cdot )$ is the weight transformation function.

The Multi-head Attention mechanism originates from the Transformer [34] model. Its core principle involves mapping query, key, and value vectors to multiple spaces through distinct linear transformations, followed by calculating the scaled dot-product attention.The computational formula is as follows:

$$\mathrm{Attention}\left(Q,K,V\right)=softmax\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)\cdot V$$

(12)

where $d_k$ represents the dimensionality of the keys, which is used to scale the dot product to prevent the issue of gradient vanishing.

Subsequently, vectors Q, K, and V of dimensionality d are transformed into single vectors of dimensionality $d/n$ using different weight matrices $W_i^Q$ , $K_i$ and $Q_i$ . Here, n denotes the number of parallel layers or heads, and these individual vectors are input into corresponding parallel attention layers.Finally, the outputs of each layer are concatenated and fused together using a Linear layer to amalgamate all head output results.

As depicted in the Multi-head Attention section in Figure 4, in our model, we harness the powerful capability of multi-head attention mechanism to capture diverse temporal scale features in time series, thereby predicting wind power sequences. The mathematical computation formula is as follows:

$$\mathrm{MHA}\left(Q,K,V\right)=\mathrm{Concat}\left({\mathrm{head}}_1,{\mathrm{head}}_2,\cdots ,{\mathrm{head}}_n\right)W^O$$

(13)

$$\begin{array}{l}{Q}_i=Q{W}_i^{{ }^Q}\\ K_i=K{W}_i^{{ }^K}i=1,2，\cdots ，n\\ V_i=V{W}_i^{{ }^V}\end{array}$$

(14)

where $W_i^Q\in {ℝ}^{d\times d_k}$ , $W_i^K\in {ℝ}^{d\times d_k}$ , $W_i^V\in {ℝ}^{d\times d_v}$ , $W^O\in {ℝ}^{n{d}_v\times d}$ , $d_k=d_v=d/n$ .

The multi-head attention architecture facilitates leveraging the complexity of input sequences and capturing long-range dependencies, thereby enhancing prediction accuracy. In this study, to mitigate overfitting, the model training employs Mean Squared Error (MSE) as the loss function and utilizes the Adam optimizer for parameter updates.

2.5. MHA-BiGRU

Although BiGRU performs effectively in handling wind power sequences, it lacks the ability to parallelize data processing, resulting in information overload and reduced computational efficiency when dealing with large datasets. To address this limitation, this study combines BiGRU with the Multi-head Attention mechanism (MHA), as depicted in Figure 4. This integrated model architecture effectively resolves the aforementioned issue. The core process involves utilizing the data trained through the BiGRU hidden layers as input for the MHA network. The decomposed Q, K, and V obtained are then fed into each head for attention value computation. Subsequently, the different results outputted from each channel are feature-weighted and concatenated through a connection layer to form the sequence.

2.6. DBO-MHA-BiGRU

The Dung Betele Optimizer (DBO) optimization algorithm, proposed by Shen et al. in 2022, introduces a novel swarm intelligence optimization approach. Its primary principle involves simulating five distinct behaviors observed in dung beetles: rolling, dancing, foraging, stealing, and reproducing, to address optimization problems. Leveraging the DBO algorithm, the MHA-BiGRU model undergoes optimization of parameters such as learning rate, number of BiGRU neurons, number of attention heads, filter count, and regularization parameters. This optimization ensures the model's convergence, with the objective of minimizing the loss function. The optimization formula is as follows:

$$f=\min (train\_loss(e\_s,h\_s,n\_l,l\_r,epoch))$$

(15)

where  $e\_s$ , $h\_s$ , $n\_l$ , $l\_r$ ,and epoch represent the number of attention heads, filter count, MHA-BiGRU model layers, learning rate, and regularization parameter, respectively. train_loss denotes the loss function during the training process.

2.7. Least Squares Support Vector Regression

Least Squares Support Vector Regression (LSSVR), proposed by Vapnik in the early 1990s, is a statistical learning method known for its fast training speed, good generalization performance, and strong ability to fit nonlinear functions. It particularly excels in handling high-frequency signals, as its core algorithm transforms the solution of a convex quadratic optimization problem into solving a system of linear equations. Consequently, LSSVR requires fewer parameters to train compared to SVR, resulting in faster training speed.

Suppose there is a training set ${\left\{\left({\mathrm{x}}_i,y_i\right)\right\}}_{i=1}^n$ , where ${\mathrm{x}}_i\in {\mathrm{R}}^d$ represents the input and  $y_i\in \mathrm{R}$ represents the output. The model calculation formula is as follows:

$$f(\mathrm{x})={\mathrm{w}}^T\varphi (\mathrm{x})+b={\displaystyle \sum _{i=1}^n{\alpha }_i\kappa ({\mathrm{x}}_i,\mathrm{x})+b}$$

(16)

In the equation, $\mathrm{w}={\displaystyle \sum _{i=1}^n{\alpha }_i\varphi ({\mathrm{x}}_i)}$ , $\mathrm{w}$ represents the weight vector, $\varphi (\cdot )$  denotes the non-linear mapping from the input space to the high-dimensional feature space, ${\alpha }_i$ stands for the Lagrange multiplier, $\kappa ({\mathrm{x}}_i,\mathrm{x})$  is the kernel function, and b is the bias term.

2.8. Regularized Extreme Learning Machine

Extreme Learning Machine (ELM) is a machine learning algorithm proposed by Professor Huang from Nanyang Technological University in 2004. Its distinguishing feature is a single hidden layer feedback neural network. It can be transformed into solving the generalized inverse problem of the M-P matrix by simply adding a least squares minimum norm problem. Consequently, ELM has fewer model parameters and boasts fast training speed. It demonstrates excellent capability in handling medium-frequency sequence information.

Suppose we have a sample set $(X_i,{\mathrm{t}}_i)$ , where the model input is denoted by  ${\mathrm{X}}_i={[x_{i1},x_{i2},\cdots ,x_{in}]}^T$ and the model output by ${\mathrm{t}}_i={[t_{i1},t_{i2},\cdots ,t_{in}]}^T$ . Then, an ELM network with L hidden layer nodes can be defined as:

$${\displaystyle \sum _{i=1}^L{\beta }_i\times g({\omega }_i\cdot x_j+b_i)}=o_j,j=1,2,\cdots ,N$$

(17)

In the equation, $g(\cdot )$ represents the activation function, ${\omega }_i$ denotes the connection weights between the output layer and the hidden layer, ${\beta }_i$ signifies the output weights between the hidden layer and the output layer. $b_i$ stands for the bias of the i-th hidden unit, and $o_j$ represents the network output. To minimize the output error, the calculation formula is as follows:  $\underset{\beta }{\min }{\Vert \mathrm{H}\beta -\mathrm{T}\Vert }_F$ , H denotes the hidden layer output and T represents the expected output.

To improve the model's generalization performance, a regularization parameter $\lambda$ is introduced to solve $\beta$ , effectively addressing numerical instability issues when computing the pseudo-inverse of H. The computation process of the RELM model, utilizing a regularized least squares method to solve $\beta$ , is mathematically expressed as follows:  $\underset{\beta }{\min }{\Vert \mathrm{H}\beta -\mathrm{T}\Vert }_F+(1/\lambda )\cdot {\Vert \beta \Vert }_F$ .

2.9. Composition of the Proposed Model

Drawing from the aforementioned methodologies, this study introduces a multi-scale hybrid wind power prediction model that integrates ICEEMDAN signal decomposition, permutation entropy (PE) reconstruction, and LSSVR-RELM-MHA-BiGRU. The model parameters are optimized using the DBO optimization algorithm. The overall model workflow, as depicted in Figure 1, is further elucidated with detailed step-by-step explanations as follows:

Step 1: ICEEMDAN decomposes the original wind power data into multiple Intrinsic Mode Function (IMF) components and a residual R. Using permutation entropy (PE), all IMF components are reconstructed to reduce computational complexity. Subsequently, the reconstructed components are categorized into high-frequency, medium-frequency, and low-frequency components based on their PE values.

Step 2: The DBO optimization algorithm is applied to optimize the hyperparameters of the MHA-BiGRU model. The optimized model is then used to predict the IMF low-frequency component after reconstruction.

Step 3: The high-frequency, medium-frequency, and low-frequency components are separately fed into the LSSVR, RELM, and MHA-BiGRU models, respectively. Predictions are obtained for each component.

Step 4: The predictions for the high, medium, and low-frequency IMF components are aggregated to obtain the final prediction result.

3. Case Study

3.1. Data Description

The wind power dataset in this paper is sourced from a wind farm operated by Elia in Belgium. To validate the accuracy of the model, four sets of power data from different seasons were selected as the original data for the model. The wind power data is recorded every 15 minutes. Details of the data are provided in Figure 5 and Table 1. During the data collection process, issues such as missing data and data errors are inevitable, which can significantly affect the accuracy of model predictions. Therefore, this paper adopts the method of removing zeros and mean interpolation to preprocess the data. Each month contains 2880 data points, and each set of data is divided into training and testing sets in a ratio of 7:3.

From Table 1 and Figure 5, it is visually evident that wind power data exhibits significant randomness and non-stationarity. Due to the large numerical values in the wind power dataset, all data is normalized through scaling to facilitate observing the data characteristics.

Subsequently, the ICEEMDAN method is employed to decompose the wind power sequence into signals (as shown in Figure 6). During decomposition, 50 instances of white noise are added, with a standard deviation (Ntsd) set to 0.2, and a maximum allowable iteration of 100. ICEEMDAN decomposes the original sequence into multiple Intrinsic Mode Function (IMF) sub-sequences. Taking the wind power sequence of March as an example, the decomposition results are illustrated in Figure 6.

As shown in Figure 6, the data is decomposed into multiple IMF components. According to the workflow diagram, it is necessary in this paper to predict each reconstructed sub-component and then aggregate the prediction results to obtain the final forecast. However, during this process, the computational complexity significantly increases, leading to substantial errors in the prediction results. Therefore, the permutation entropy (PE) method is employed to calculate the entropy value of each sub-component separately, re-quantifying the complexity of each sub-component. Simultaneously, all sub-components are reconstructed into three new sub-components: high-frequency, medium-frequency, and low-frequency, reducing the sequence complexity. The PE values of each sub-component are shown in Table 2. Based on the entropy values, IMF1-IMF3 are selected as high-frequency components, IMF4-IMF9 as medium-frequency components, and IMF10 as the low-frequency component. The reconstructed new sub-components are illustrated in Figure 7.

3.2. Performance Metrics

To better assess the performance of the model predictions, this study employs three different error evaluation metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE).

$$e_{RMSE}=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\left|f\left(x_i\right)-a\left(x_i\right)\right|}^2}}{n}}$$

(18)

$$e_{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|f\left(x_i\right)-a\left(x_i\right)\right|}$$

(19)

$$e_{MAPE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{f\left(x_i\right)-a\left(x_i\right)}{a\left(x_{ }\right)}\right|\times 100\%}$$

(20)

The equation represents the calculation of the Mean Absolute Percentage Error (MAPE) metric, where $a\left(x_i\right)$ denotes the actual observed value of the i-th training sample, $f\left(x_i\right)$ represents the predicted value of the i-th sample, and n represents the total number of samples.

4. Comparisions Results

Experiment and Results Analysis

This study employed nine models to forecast wind power sequences for four different seasonal months, as outlined in Table 3. A comparative analysis was conducted between the proposed ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model and other models including LSSVR, RELM, BiGRU, MHA-BiGRU, ICEEMDAN-LSSVR, ICEEMDAN-RELM, ICEEMDAN-MHA-BiGRU, and ICEEMDAN-LSSVR-RELM-MHA-BiGRU. The evaluation was performed using three different metrics: MAE, RMSE, and MAPE. The results were visually presented using bar charts, radar charts, and stacked bar charts, providing a comprehensive comparison of the predictive performance of the models, as depicted in Figure 8 and summarized in Table 4.

Upon careful examination of Table 4 and Figure 8, the following conclusions can be drawn:

1. Comparison of LSSVR, RELM, and BiGRU models suggests that multiscale hybrid models outperform single-scale models in wind power sequence prediction.

2. Evaluation of models BiGRU and MHA-BiGRU reveals a notable decrease of 18.72% in RMSE and 7.08% in MAE for the May forecast. Additionally, for the March forecasts, RMSE values decrease by approximately 10%, indicating that the incorporation of multi-head attention mechanisms enhances predictive accuracy.

3. The inclusion of decomposition algorithms generally enhances predictive performance compared to single models in wind power prediction. For example, in the August metrics, models LSSVR, RELM, and BiGRU incorporating the ICEEMDAN decomposition algorithm exhibit reductions of 25.57%, 25.29%, and 27.63% in RMSE values, respectively, along with approximately 20% decreases in other metrics. This underscores the effectiveness of models incorporating decomposition algorithms in improving prediction accuracy.

4. Comparison between Model 8 and single-scale models combined with decomposition algorithms indicates that hybrid algorithms generally exhibit superior predictive performance. For instance, in the radar chart of MAPE values in Figure 7, the proposed model consistently exhibits the lowest values along its axes. Furthermore, in terms of RMSE values for March, Model 8 demonstrates a 57.20% reduction compared to Model 5. This underscores the advantage of multiscale hybrid algorithms in achieving smaller errors and higher predictive accuracy.

5. Evaluation of Model 8 against the proposed model highlights improvements in predictive performance with the introduction of DBO optimization algorithms. For instance, in the November forecast, there are reductions of 25.59%, 46.40%, and 28.96% in RMSE, MAE, and MAPE values, respectively. This indicates that the incorporation of DBO optimization algorithms enhances predictive performance, rendering the model proposed in this study more suitable for wind power sequence prediction.

As shown in Figure 6, the data is decomposed into multiple IMF components. According to the workflow diagram, it is necessary in this paper to predict each reconstructed sub-component and then aggregate the prediction results to obtain the final forecast. However, during this process, the computational complexity significantly increases, leading to substantial errors in the prediction results. Therefore, the permutation entropy (PE) method is employed to calculate the entropy value of each sub-component separately, re-quantifying the complexity of each sub-component. Simultaneously, all sub-components are reconstructed into three new sub-components: high-frequency, medium-frequency, and low-frequency, reducing the sequence complexity. The PE values of each sub-component are shown in Table 2. Based on the entropy values, IMF1-IMF3 are selected as high-frequency components, IMF4-IMF9 as medium-frequency components, and IMF10 as the low-frequency component. The reconstructed new sub-components are illustrated in Figure 7.

To further validate the accuracy of the proposed model, Figure 9 illustrates the comparison of forecasts between the ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model and other models for four months. Figure 10 displays the relative error plots for BiGRU, MHA-BiGRU, ICEEMDAN-MHA-BiGRU, ICEEMDAN-LSSVR-RELM-MHA-BiGRU models, and the proposed model. Figure 11 presents linear regression plots with 95% confidence intervals for the true values and predictions of these five models.

Upon closer examination of the magnified portions in Figure 9, it is evident that models incorporating decomposition algorithms and multiscale hybrid models exhibit the highest degree of fitting. This underscores the effectiveness of the proposed model. From Figure 10, it can be observed that the proposed method demonstrates the smallest fluctuation range in relative errors. Moreover, in Figure 11, the linear regression confidence band for the proposed model is the narrowest. Particularly, after incorporating attention mechanisms and decomposition algorithms, the scatter plots become more concentrated, and the confidence band notably narrows. Based on the aforementioned analysis, it can be concluded that the ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model proposed in this study exhibits effectiveness, applicability, and yields satisfactory prediction results in wind power forecasting.

5. Conlcusions

This study proposes a novel multiscale wind power prediction model that integrates ICEEMDAN, PE, LSSVR, RELM, Multi-head Attention, and BiGRU methods. By leveraging wind power data collected from a Belgian Elia wind farm over four months, the model's predictive performance advantages are thoroughly assessed and validated through various analytical approaches, including error metrics, relative error plots, and linear regression analyses. Comparative ablation experiments are conducted against a range of models including LSSVR, RELM, BiGRU, MHA-BiGRU, ICEEMDAN-MHA-BiGRU, and ICEEMDAN-LSSVR-RELM-MHA-BiGRU. Key findings from this analysis are summarized as follows:

1. The multiscale model, which integrates LSSVR, RELM, and BiGRU methods, outperforms single-scale prediction models in terms of predictive accuracy.

2. Compared to the ICEEMDAN-MHA-BiGRU model, the proposed ICEEMDAN-LSSVR-RELM-DBO-MHA-BiGRU model demonstrates superior fitting accuracy across different wind power sequence prediction plots for the four data sets. This underscores the effectiveness of decomposing wind power sequences into high, medium, and low-frequency components in enhancing predictive accuracy.

3. Employing permutation entropy to reconstruct sub-components of decomposed sequences into multiple frequency bands and utilizing multiscale models to predict each frequency band contributes significantly to improved predictive performance.

4. The predictive capability of BiGRU and MHA models optimized using the DBO algorithm surpasses that of models without DBO optimization. This indicates that introducing optimization algorithms can further enhance wind power prediction accuracy.

6. Further Research

The multiscale hybrid wind power prediction model proposed in this study advances the capability of single-step prediction in wind power forecasting, offering valuable insights for the rational planning of power systems. However, there are avenues for improving prediction methods, which include:

1. This study relies solely on historical wind power data without considering additional factors such as geographical conditions and turbine statuses, which can significantly influence wind power prediction accuracy. Thus, future research could explore integrating multiple factors to enable multi-step prediction.

2. The dataset used in this study is limited to a single wind farm, which may restrict the model's ability to generalize across different environments. Future endeavors should aim to validate the proposed model using data from multiple wind farms to enhance its robustness and applicability.