Machine Learning Dynamic Ensemble Methods for Solar Irradiance and Wind Speed Predictions

1. Introduction

The Electricity generated by fossil fuel sources has been the main driver of climate change, probably over 70% of greenhouse gas emissions and over 90% of all carbon gas emissions. The alternative of decarbonizing the world’s electricity generation system is a trend focused on alert sources of renewable energy, whose generation costs are increasingly accessible [1].

A very undesired effect when it comes to electrical generation from alternative resources is the impact of intermittency generation on the electrical grid, since this generation is dependent on weather conditions, and one of the means to eliminate or reduce its uncertainties is the availability prediction of these resources [2].

The influences of atmospheric factors on the generation of electrical energy from solarand wind sources are usually the main problem in the generation of smart grids, where large-scale generation plants need to be integrated into the electrical grid, which directly affects planning, investment, and decision-making purposes. Forecast models can minimize that problematic through machine learning models [3].

The benefits of optimizing the forecast of generation from wind and solar sources using models is also an economic factor, as it gives greater security to the electricity sector through the improvement of renewable energy purchase contracts [4].

A 14-year-long data set was explored in [5], containing daily values of meteorological variables. It was used to train three artificial neural networks (ANNs) in several time horizons to predict the global solar radiation for Fortaleza, in the Brazilian Northeastern region. The accuracy of the predictions was considered excellent according to its normalized root-mean-squared error (nRMSE) values and good relative to mean absolute percentage error (MAPE) values.

The variability of mathematical prediction models has individual importance inherent to each one of the methods employed and, in this scenario, dynamic ensemble models emerge, which present potential better performance when compared to individual models, since they seek maximum optimization by considering the best of the individual models. This approach is currently very successfully used in research and industrial areas. Several dynamic ensemble methods have been developed for forecasting energy generation from renewable sources in which they use the presence of well-known forecast models such as random forest regression (RF), support vector regression (SVR), and k-nearest neighbors (kNN), which are applied to integrate optimizations for use in dynamic ensemble methods [6].

The Support Vector Machine was first developed for classification models and is largely discussed [7,8], in recent approaches [9] to develop a novel method for the maximum power point tracking of a photovoltaic panel and in [10], where it is discussed the solar radiation estimation by five different machine learning approaches.

The KNN method predicts a new sample using the K-closest samples. Recently this approach has been used in [11], where Virtual Meteorological Masts use calibrated numerical data to provide precise wind estimates during all phases of a wind energy project to reproduce optimal site-specific environmental conditions.

Most studies have focused on accurate wind power forecasting, where random fluctuations and uncertainties involved are considered. The study in [12] proposes a novel ultra-short-term probabilistic wind power forecasting using an error correction modeling by the random forest approach.

The Elastic Net is a regularized regression method that linearly combines the penalties of the LASSO and Ridge methods. In [13] the study uses forecast combinations that are obtained by applying regional data from Germany for both solar photovoltaic and wind through the Elastic Net model, with cross-validation and rolling window estimation, in the context of renewable energy forecasts.

The state-of-the-art uses dynamic ensemble methods in a meta-learning approach such as arbitrating, which combines the output of experts according to predictions of the loss they will incur, and also windowing approaches, which have parameterizations for adjusting the degree of data to be considered [14].

In [15], the global climate model (GCM) is studied to improve a near-surface wind speed (WS) simulation through 28 coupled model intercomparison using dynamical components.

In [16], a hybrid transfer learning model based on a convolutional neural network and a gated recurrent neural network is proposed to predict short-term canyon wind speed with fewer observation data. The method uses a time sliding window to extract time series from historical wind speed data and temperature data of adjacent cities as the input of the neural network.

In [17] is explored the Multi-GRU-RCN, an ensemble model to get significant information such as precipitation and solar irradiation through short-time cloud motion predictions from a cloud image. The ensemble modeling used in [18] integrates wind and solar forecasting methodologies applied to two locations with different latitudes and climatic profiles. The obtained results reduce the forecast errors and can be useful in optimizing the planning for using intermittent solar and wind resources in the electrical matrices.

A proposed new ensemble model in [19] was based on Graph Attention Network (GAT) and GraphSAGE to predict wind speed in a bi-dimensional approach using a Dutch dataset considering several time horizons, timelags, and weather influences. The results showed that the ensemble model proposed was equivalent to or outperformed all benchmarking models and had smaller error values than those found in reference literature.

Under a 5 min time-step, in [20] it was applied time horizons ranging from 5 min to 30 min in evaluating the solar irradiance short-term forecasts to Global Horizontal Irradiance (GHI) and Direct Normal Irradiance (DNI) using deep neural networks with 1-dimensional convolutional neural network (CNN-1D), long short-term memory (LSTM), and CNN-LSTM. The metrics used were the mean absolute error (MAE), mean bias error (MBE), root-mean-squared error (RMSE), relative root mean squared error (rRMSE), and coefficient of determination (R²). The best accuracy was obtained for a horizon of 10 min, improving 11.15% on this error metric compared to the persistence model.

In front of this, the main contribution of this article is to demonstrate the influence of dynamic ensemble arbitrating and windowing methods on machine learning methods traditionally used for predicting electrical generation. We also present their greater efficiency, using data of interest for energy production with input variable of wind speed and solar irradiance, used, respectively, for both wind and solar farms. We have followed this approach because of its advantage in exploring dynamic ensemble methods, since these seek the best pre-existing efficiency for generating a unique and more effective predictability model.

2. Location and data

In this paper, two data types were used to carry out the analysis, which were acquired from solarimetric and anemometric station located in Petrolina – PE. The data were collected from the SONDA network (National Organization of Environmental Data System) [21], which was a joint collaboration of several institutions and was created for the implementation of physical infrastructure and human resources, aiming at raising and improving the database of solar and wind energy resources in Brazil.

In Table 1, information on the solarimetric and anemometric station can be found, and its location on the map is shown in Figure 1.

2.1. Wind speed data

The wind speed data was obtained in m/s from a meteorological station, which has anemometric sensors at altitudes of 25 m and 50 m from the ground. The highest altitude was chosen for this study, aiming to reduce the effects of the terrain and closer to the altitudes currently in practice for wind turbines [22].

2.2. Irradiance data

The Global Horizontal Irradiance (GHI) data acquired from the solarimetric station were used in this study, and the clear-sky coefficient was considered, in order to remove dependence on air mass in the irradiance values that reach the sensors [23], through the use of the clear-sky factor (I_cs) [24], using the polinomial fit model [27]. The work [25] obtained promising results from the same database using two machine learning estimation models for (GHI).

In order to obtain irradiance data independent of air mass variations, we used k_t, which is defined by the ratio between the Global Horizontal Irradiance value (GHI) (I) and clear sky factor (I_cs), as shown in Equation 1.

$$k_t=\frac{I}{I_{cs}}$$

(1)

3. Methodology

Initially, wind speed and irradiance data were acquired and the intervals for the test and training sets were determined. For wind speed data, in a measurement period from 2007 to 2010, the first three years were used as the training data set and the last year as the test set. In order to allow the evaluation of the performance of the tested forecasting models and also of dynamic ensemble methods, a computational code was developed in Python to evaluate the output values obtained by the well-known Machine Learning forecasting methods: Random Forest, k- Nearest Neighbors (kNN), Support vector Regression (SVR), and Elastic Net. For each of the methods, the best performance parameters (lower Root Mean Squared Error (RMSE)) were evaluated. Right after the stage of acquisition and determination of the optimal parameters for each of the models, the methods of dynamic ensemble windowing and arbitrating were executed, from which performance metrics values were also obtained: Coefficient of Determination (R²), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). These values were compared to evaluate the efficiency of the dynamic ensemble methods compared to other stand-alone models. It was also evaluated the variation of the lambda parameter for windowing, which is the length used for the extension of the values considered in the data forecast. The methodology used can be seen in Figure 2.

In the data pre-processing, a recursive approach of Lagged Average values for k_t and ν time series was applied: this feature is given by the vector L(t) with components calculated using Equation (2).

$$L_i(t)=\frac{1}{N}\sum _{t\in ]t-i\delta -T,t-(i-1)\delta -T]}^{}\mathrm{x}\left(t\right)$$

(2)

3.1. Windowing method

The diversity of the models makes the forecast analysis rich and complex, since each model has strong points and other weaknesses, in the sense that from this combination, the best results can be treated and considered to obtain more accurate forecasts. To perform this combination, it is necessary to know how to estimate at which points certain specific models perform better.

Windowing [14] is a dynamic ensemble model, where weights are calculated based on the performance of each individual model, evaluated in a data window referring to immediately previous data. The size of this window is parameterized by the λ value. This means that the weights of each model are re-evaluated at each time step, and then they are classified to catalogue only the best performance results, generating a hybrid model.

3.2. Arbitrating method

Arbitrating [28] uses the metalearning method strategy to learn and predict the classifiers. In this study it regards the weights based on each model’s performance for a given time step. At each simulation instant, the most reliable model is selected and included in the prediction process.

3.3. Machine learning prediction models and dynamic ensemble method parameters

In the data training stage, GridSearch was used with 5-fold cross validation. The search parameters are shown in Table 2.

3.4. Performance Metrics Comparison Criteria

As the purpose of this work is to evaluate the performance of dynamic ensemble methods against other methods, performance metrics had to be determined to allow it. The metrics used were those of Equations 3, 4, 5 and 6.

• Coefficient of Determination (R²)

$$R=1-\frac{{\sum }_{i=1}^N(y_i-\widehat{y_i})}{{\sum }_{i=1}^N(y_i-\overline{y_i})}$$

(3)

• Root Mean Squared Error (RMSE)

$$RMSE=\sqrt{\sum _{\begin{array}{c}i=1\\ \end{array}}^N(\widehat{y_i}-y_i)}$$

(4)

• Mean Absolute Error (MAE)

$$RMSE=\frac{1}{N}\sum _{\begin{array}{c}i=1\\ \end{array}}^N\left|\widehat{y_i}-y_i\right|$$

(5)

• Mean absolute percentage error (MAPE)

$$MAPE=\frac{1}{n}\sum _{\begin{array}{c}i=1\\ \end{array}}^n\left|\frac{y_i-\widehat{y_i}}{y_i}\right|$$

(6)

4. Results and Discussion

Discussions are carried out around the results generated regarding the analysis of efficiency metrics for the machine learning methods discussed here, in an attempt to determine which method/parameters obtains the best performance in the application for wind speed and solar irradiance data.

4.1. Wind Speed Predictions

During the search for best-performance methods, the optimized parameters for each of the tested methods needed to be known. This allows the elaboration of the dynamic ensemble, which is built upon the merging of the best-performance results at each time step and for all the methods in question. The optimal parameters found for each of the time horizons are shown in Table 3.

Efficiency evaluations for each of the forecasting methods were based on performance metrics evaluations for each time horizon under study (t+10, t+20, t+30 and t+60). Initially, for all time horizons, windowing proved to be the most efficient method. Then, a fine-tuning evaluation was performed based on the variation of the windowing parameter to assess its influence on performance. The predominance of better performance for windowing in all time horizons and its comparisons can be seen in Table 4 and Figure 3.

Elastic Net is a penalized linear regression model that is a combination of Ridge and Lasso regression into a single algorithm and uses best_l1_ratio as a penalty parameter during the training step, being 0 for Ridge and 1 value for Lasso regression. From Table 3, the parameter obtained the value of 1, which means that Lasso regression was used in its entirety.

As with the evaluation carried out by the RMSE, the values of R², MAE and MAPE were also assessed. Once the best performance was found for the windowing ensemble method, an in-depth analysis was performed based on the variation of its parameter ʎ to assess the influence on its internal performance. Since the time horizon that presented the best performance was t+10, this was the focus of the analysis, as shown in Figure 4, Figure 5, Figure 6 and Figure 7. The detailed data for all the horizons is shown in Table 5, Table 6 and Table 7.

When we check the influence of the ʎ parameter on the windowing method performance, it was found that from ʎ = 74 it is no longer the most efficient method, where SVR becomes the best one, due to its lowest RMSE value. Anyway, it is important to highlight that the best performance value for the windowing method, which is the best performance overall, was found for ʎ = 19. The performance comparison between the two methods can be seen in Figure 8.

4.2. Irradiance Predictions

During the search for best-performance methods, the optimized parameters of each of these methods needed to be known to allow the elaboration of the dynamic ensemble, which is built from merging the best-performance results at each instant and for each of the methods in question. The optimal parameters for each time horizon are shown in Table 8.

Efficiency evaluation for each of the solar irradiance forecasting methods were based on performance metrics for each time horizon under study (t+10, t+20, t+30 and t+60). Again, windowing proved to be the most efficient method for all time horizons, with the best method being found for the t+10 time horizon, having the lowest RMSE value, using its parameterizations with ʎ = 50 initially. Then, fine-tuning was performed based on the variation of the windowing parameter to assess its influence on performance. The predominance of better performance for windowing in all time horizons and its comparisons can be seen in Table 9 and Figure 9.

Just like the evaluation carried out by the RMSE, the values of R², MAE, and MAPE were also analyzed. After the best performance was found for the windowing method, an in-depth analysis was performed based on the variation of its parameter ʎ to assess the influence on its internal performance. Since the time horizon that presented the best performance was t+10, this was the focus of the analysis, as shown in Figure 9, Figure 10, Figure 11 and Figure 12. The detailed data for all tested time horizons is shown in Table 10, Table 11 and Table 12.

Some authors applied Elastic Net in time-varying combinations [13], using RMSE as a performance metric. They found that, for PV forecasts, it has been reached 13.4% more precise forecasts than the simple average and for the wind forecast, it has been reached 6.1% better forecasts.

In [18] an ensemble method was studied, which used MAPE as the comparative efficiency metric for wind speed data, with a value of 9.345% and solar with 7.186%, which proved to be the most efficient.

In this study, performance improvements were obtained for the most efficient method (windowing) compared to the second most efficient for wind speed of 0,56% and, for solar irradiation, 1.86%.

4.3. Comparison with results from the literature

The performance of the windowing approach was compared with other wind forecasting models found in the literature. It is important to disclose that a direct comparison between different predictive models is not an easy task, once each applied approach has its own objectives, hyperparameters, and input data [19].

The results found in literature for wind speed forecasting are compiled and presented in Table 13, where RMSE and MAE are in m/s.

Analyzing the results for reference [19], in which wind speed was forecasted for the Netherlands using an ensemble approach merging graph theory and attention-based deep learning, we can observe that the proposed windowing ensemble model is not able to surpass the results for RMSE nor MAE for t+60 forecasting horizon. The accentuated difference between these two models can be explained because the GNN SAGE GAT model, being developed to handle graph-like data structure, excels in retrieving complex spatiotemporal relationships underlaying the dataset, drastically improving its forecasting capacity when compared with other ML and DL models alike.

In reference [29], the authors proposed a wind forecasting for a location in Sweden, with a model based on a bi-directional recurrent neural network, a hierarchical decomposition technique, and an optimisation algorithm. When compared with their results, the windowing model proposed in this paper offer improvement over the reference results for t+10 forecasting horizon by 1% and by 20% for t+60. Analysing MAE and MAPE, the windowing indicates improvement over these metrics for t+10 and t+60, increasing by 28% the MAE value for t+10, and 9% for t+60. Regarding MAPE, the improvement is 64% for t+10 and 95% for t+60.

In the work of Liu et al. [31], other deep learning-based predictive model was proposed. It used a hybrid approach composed of data area division to extract historical wind speed information, and an LSTM layer optimized by a genetic algorithm to process the temporal aspect of the dataset to forecast wind speed in Japan. Compared to this reference, the windowing model showed no improvement for wind speed forecasting. However, the windowing approach offers competitive forecasting for the assessed time windows, being in the same order of magnitude as the ones in the reference. In work [32], the authors proposed the employment of another hybrid forecasting architecture composed of CNN and LSTM deep learning models for wind speed estimation in the USA. Their results, when compared against the windowing methodology, are very similar for all forecasting horizons, showing that both windowing and CNN-LSTM offer good results for wind speed estimation for these time intervals.

In Dowell et al. [30], a statistical model for estimation of future wind speed values in the Netherlands was proposed. For the available t+60 time horizon we observe that, again, the forecasted wind speed for the reference and proposed windowing models are very similar, deeming both models as valuable tools for wind speed forecasting.

For GHI forecasting, the results found in the literature are presented in Table 14.

In work [20], the deep learning standalone model of CNN was applied to estimate future GHI values in the USA. Comparing the GHI forecasting results achieved by windowing with this reference, we observe that the proposed model was not able to provide superior forecasting performance. However, the windowing results are still competitive since both approaches were able to reach elevated coefficient of determination values for all the assessed forecasting horizons, with a slight advantage for the deep learning model.

In reference [35], the authors combined principal component analysis (PCA) with multivariate empirical model decomposition (MEMD) and gated recurrent unit (GRU) to predict GHI in India. In their methodology, the PCA extracted the most relevant features from the dataset after it was filtered by the MEMD algorithm. Lastly, the future irradiance was estimated by the deep learning model of GRU. Compared to their approach, the windowing model could not improve the GHI forecasting considering t+60 time window. Also, the reference model MEMD-PCA-GRU provided elevated R² value of 99%, showing clearly superior performance over the proposed ensemble model.

Compared with the physical-based forecasting models proposed in [36] and [37], we can conclude that windowing can achieve similar results for time horizons of t+30 and t+60. In [36], authors used the FY-4A-Heliosat method for satellite imagery to estimate GHI in China. Although the windowing model could not improve the GHI forecasting for t+30 and t+60 time windows, the proposed model is able to return relevant results for irradiance estimation in both cases. The second physical-based model proposed in [37] was applied to estimate GHI in Finland. In their methodology, the Heliosat method is again employed, together with geostationary weather data from satellite images. Compared to their proposed approach, the windowing model can improve the GHI forecasting for t+60 in 8%, providing significant advance in the irradiance estimation.

In work [33], the authors used the state-of-the-art transformer deep learning architecture together with sky images [34] to GHI estimation in the USA. Analyzing their results and the ones provided by the windowing method, we observe that the transformer-based model reaches the best GHI forecasting values for RMSE in all the assessed time windows.

After the comparison of the ensemble windowing approach with reference models found in the literature, we see that wind speed prediction is often competitive and usually improves wind speed prediction for the assessed forecasting horizons. The results for wind speed prediction using the ensemble model corroborate the results found in the literature, where the ensemble approach often reaches state-of-the-art forecasting in time-series prediction applications [18,38,39,40]. Their improved performance comes from the combination of weaker predictive models to improve their overall forecasting capacity, also reducing the ensembled model’s variance [41,42].

However, the proposed dynamic ensembled approach faced increased difficulty when determining future GHI values. This may be an indication that the irradiance forecasting is a more complex non-linear natural phenomenon, requiring improved extraction of spatiotemporal information from the dataset. Since the proposed ensemble model does not have a deep learning model in its architecture it cannot properly identify and extract spatiotemporal information underlying the dataset, lacking in providing better irradiance estimation. Deep learning model can often excel in this type of task, as proved in the results from Table 14. Extensive literature can be found regarding improvements of time-series forecasting problems when complex and deep approaches are employed [19,20,43,44].

5. Conclusions

This work proposes to evaluate the performance of two ML methods of dynamic ensemble, using wind speed and solar irradiance data separately as inputs. Initially, wind speed and irradiance data from the same meteorological station were collected, the time horizons to be studied were determined (t+10 min, t+20 min, t+30 min and t+60 min), then a recursive approach of Lagged Average values was applied to evaluate the models’ predictors.

ML methods well known in other energy forecasting works and applied to wind and irradiance data were selected to compare efficiency with two other methods that use a dynamic ensemble approach (windowing and arbitrating). The programming code in Python was developed to catalog the optimal efficiency parameters of each previously known model, based on error metrics and coefficient of determination. The dynamic ensemble methods (windowing and arbitrating), based on the optimal parameters of each previously calibrated models (Random Forest, k-Nearest Neighbors, Support Vector Regression, and Elastic Net), generated a single model with greater efficiency for both wind and solar irradiance data.

For forecasting wind speed data, the most efficient method was found to be windowing for all time horizons, when evaluated by the criterion of the lowest RMSE value, and specifically for the time horizon t+10, as evidenced in Figure 3. The greater efficiency was found in an interval of 1 to 74 for the ʎ parameter, reaching maximum performance for the value ʎ = 19, as seen in Figure 8, which suggests that the windowing parameterization directly influences the method's performance.

For the solar irradiation forecasting, the most efficient method was also windowing and the t+10 min time horizon reached the lowest RMSE value. Differently from what was found for the wind speed data, a greater linearity in the trend was perceived from the ʎ windowing parameter variation graph when analyzing its RMSE values. Looking for the ʎ interval under study, the best performance value (using RMSE criteria) of ʎ = 1 was found, as can be seen in Figure 10.

Using wind speed data, the efficiency gain of the most efficient model (windowing for the time horizon t+10 min and ʎ = 19, see Table 4), when compared to the second highest efficiency (SVR) was 0.56%, when using the lowest value RMSE metric. A similar trend could be observed for the model using solar irradiance data. The efficiency increase, comparing the most efficient model (windowing for the time horizon t+10 min and ʎ = 1, see Table 9) to the second highest efficiency (Arbitrating) was about 1.72%, and when compared to the third most efficient method (SVR), it was about 1.96%.

Also, extensive comparisons with spatiotemporal models found in the literature show that the dynamic ensemble model for wind speed often provides superior forecasting performance for the assessed time horizons, deeming the proposed approach as a valuable tool for wind speed estimation. Regarding irradiance forecasting, the dynamic ensemble architecture proposed in this study could not surpass the deep learning-based models, which showed superior spatiotemporal identification, and consequently better estimated GHI values. However, the proposed windowing approach can provide competitive results and superior GHI forecasting when compared to physics-based predictive models.

For future works, the dynamic ensemble architecture can be improved with the addition of more complex machine learning models, such as deep learning and graph-based approaches. This may boost the windowing forecasting capacity for GHI and wind speed estimation once it will be able to benefit from spatiotemporal data information underlying the dataset. The development of an ensemble model able to provide accurate and precise estimations can then be employed in the development of real-time forecasting applications, helping the evaluation of wind and solar farm installation.