Multimodel Forecasting of Hourly Electricity Demand

1. Introduction

Planning the generation (supply) of energy on the market, regardless of its scope, requires demand forecasts. Forecasting the electricity load is the subject of many research in literature. In the energy market, regardless of the perspective (long-term, e.g. annual, or short-term, e.g. hourly or daily), forecasting the demand for electricity is of key importance – it also requires different approaches depending on the forecast horizon. The forecasting methods proposed in the literature use different types of data, including historical data, which are characterized by cyclical behaviour and trends, which supports the assumption that demand will follow a similar pattern. The specificity of this market – the stochastic nature of electricity demand, together with parameters such as weather conditions (including temperature, humidity, wind speed, cloud cover), time of day or day of the week (holidays, mass events) and work mode – affect deviations from the expected demand. Forecasting the demand for electricity is important, among other things, in capacity and transmission planning, generation planning and also in pricing. For capacity planning, it is necessary to make a long-term estimate of the total demand as a function of economic parameters and those resulting from demographic conditions, while short-term (hourly) estimates allow for improving the efficiency of day-ahead markets.

The importance of forecasting in the energy market is crucial. Review studies are available in the literature on the subject. Reviews are carried out of methods for long-term electric load forecasting [1], methods, determinants, and policy implications of forecasting electricity demand [2], electrical load forecasting models [3], electricity demand forecasting techniques [4], energy models for demand forecasting [5]. A detailed analysis of the literature on forecasting methods is presented in [6]. In addition, a review of research in the field of energy price forecasting was carried out by [7], highlighting the complexity of the available solutions, their strengths and weaknesses, and the opportunities and threats associated with forecasting tools. Hong and Fan [8] reviewed issues related to probabilistic forecasting of electricity demand, covering the most important techniques, methodologies and assessment methods. They also pointed to the need for further research, including the development of repeatable case studies, the improvement of methods for assessing and valuing probabilistic forecasts, and the consideration of new technologies and energy policy in the probabilistic forecasting process.

The issue of territory plays an important role in forecasting electricity demand. Forecasts may concern areas of varying scope (diversity), as well as facilities with different characteristics. Below is a list of selected studies that concern the area, among others:

1/ Countries, including, for example: Italy [9,10], Jordan [11], Spain [12], China [13,14], Turkey [2,6,15,16], Australia [17].

2/ Regions, such as the state of Victoria in Australia [18], Taiwan [19], England and Wales [20].

3/ Larger areas, such as Europe [21], selected EU countries (Austria, Germany, Spain, France, Italy, Great Britain) and US states (Florida, New York, California) [22] , groups of countries: Poland, France, Germany, Great Britain, Turkey and the Scandinavian countries [23], Great Britain and France [24].

4/ Smaller territories, such as local areas in Denmark [25], Prague in the Czech Republic [26], and the Green Energy Park in Morocco [27].

5/ Urban units, such as, for example: public buildings [28], residential buildings [29,30].

Due to the specific nature of the energy market, hourly forecasts are developed [15,23,24,31,32,33], as well as forecasts of daily demand [16,17,18,27,34,35,36,37], and forecasts for longer periods, e.g., weekly [23], monthly [28,38,39], or annual [23,40]. Numerous research methods and approaches are employed to develop these forecasts.

In studies concerning the assessment of demand and load (electric load forecasting), a wide variety of forecasting methods are used, including:

• artificial neural networks [28], deep CNN [18]);

• seasonal hybrid procedure [13];

• particle swarm optimization [41];

• modelling – SARIMAX with interactions [33], application of residual modification approach in seasonal ARIMA [14], a semi-parametric additive model [42], Generalized Additive Model (GAM) [17], hybrid RSVMG model (recurrent support vector machines with genetic algorithms) [19], an adaptive neural-wavelet model [43];

• a model based on support vector regression with differential evolution algorithm [40];

• using double seasonal exponential smoothing [44];

• expands double-seasonal methods – specifically double seasonal ARMA and adaptations of Holt-Winters exponential smoothing – to handle the triple-seasonality structure [45];

• with exponentially weighted methods – compares exponential smoothing formulations, discount weighted regression, cubic splines, and singular value decomposition (SVD) [24];

• spectral analysis using the fast Fourier transform (FFT) [23];

• integration of artificial neural networks and genetic algorithm [46].

In their study, Vilar, Cao, and Aneiros [35] used a functional approach (FDA), in which the entire curve of the variable during the day (demand or price) is treated as a single functional observation, as well as non-parametric and semi-parametric models, in which non-linear functional autoregression and semi-functional partial linear (SFPL) models were used.

In forecasting electricity demand, the potential benefits of using numerical weather forecasting models are also assessed [9], as are the impact of calendar effects and the detail of the forecast [29], periodic variations and demand segregation [6], and the assessment of demand by breaking down the aggregated (total) electricity demand into components such as heating, cooling, lighting and industrial activity [23]. To predict electricity consumption, the following are used, among others: Multi-Scale Graph Attention Network Based on Encoding Decomposition [47].

The multitude of methods used allows researchers to perform comparative analyses, including, for example, the comparative analysis of Kolmogorov-Arnold and recurrent neural networks for day-ahead photovoltaic forecasting [27]. The effectiveness of the DCNN architecture was compared with other machine learning models and recurrent networks (RNN, ELM), demonstrating higher accuracy and effectiveness in mapping nonlinear dependencies in the data [18].

Various factors influencing demand are also assessed, such as holiday periods [48] and meteorological variables [17,39].

There are also studies on forecasting demand/consumption/management in the field of energy from renewable sources [49,50,51].

The aim of this paper is to propose the model for hourly electricity load forecasting using the combination of five separate models: regression analysis, ARIMA, exponential smoothing, support vector regression and artificial neural networks. In the final step, forecasts from these models are combined through the multiple regression.

Calculations presented in this paper were performed using STATISTICA ver. 13, Optuna library, and Python (scikit-learn).

2. Materials and Methods

The process of short term forecasting the electricity load heavily depends on the organization of the order system in a given company. Our data comes from the area in the southern Poland. A certain number, unknown to us, was added to all values making the data anonymous. In the company which owns the real data, staff starts work at 6 a.m. The available data covers the period until last midnight. The staff has to order electricity for the next day until 11 a.m. It means that the forecasting system should provide predictions for the next 48 hours, starting midnight. Practically prediction for 25-48 hours are really important since the load for the current day has been ordered the day before. We assume that weather forecast for the day ahead is available.

Models, results and propositions are based on many exercises, different versions and also different data. What we proposed here is somehow the final results of our investigations.

The learning set of hourly data is a rolling one. Each day new 24 observations are added, and the oldest 24 are removed, so the learning set has always the same length. The following individual models are estimated on the same learning set:

• multiple regression plus ARIMA of residuals (MR+ARIMA);
• ARIMA;
• Exponential Smoothing (ES)+ARIMA;
• Support Vector Regression (SVR)+ARIMA;
• Artificial Neural Network (NN)+ARIMA.

In the first part of the analysis, structures of the individual models are established. They are unchanged in time, during the rolling process. This assumption has been validated through our analyses, but of course these structures should not be stable forever. It is difficult to advise how often the model structure should be checked. It heavily depends on individual cases – areas, seasons of the year, and maybe some other factors.

MR+ARIMA

This approach has two steps. In the first one, multiple regression model is estimated, and in the second one, residuals are checked. If they are not purely random, then the ARIMA model is fitted to them. The final theoretical values and forecasts are calculated as sum of MR and ARIMA (if estimated).

The list of explaining variables in multiple regression models consists of one measurable external variable – average daily temperature and dummy variables identifying the day of the week and the hour of the day. Dummy variables are coded as 0/1. There are 6 variables for week days and 23 variables for hours. The missing ones – baselines – represents the minimum load. There is one more dummy variable identifying local holidays which comes not on Sundays – like Christmas, National Holidays, bank holidays all others of this type. Possible trend is covered by variable t. Multiple regression model is estimated by Ordinary Least Squares method. ARIMA of residuals is identified, estimated and verified in a classical way as described in the next paragraph.

ARIMA

Autoregressive Integrated Moving Average (ARIMA) models are widely used in the analysis and forecasting of different types of time series, including the ones with trend and seasonality. Electricity load time series always have a short-period seasonal component and usually no trend. The model which is used in this paper is sometimes called SARIMA, as Seasonal ARIMA (Just ARIMA is used in STATISTICA software). No matter which notation is used, the model has two components – nonseasonal and seasonal. It is defined in two brackets as ARIMA(p,d,q)(Ps,Ds,Qs), where:

• p – order of autoregression;
• d – number of first order differencing;
• q – order of moving average;
• s – seasonal period;
• Ps – order of seasonal autoregression;
• Ds – number of seasonal differencing;
• Qs – order of seasonal moving average.

In classical approach orders of autoregression and moving average are identified through the analysis of autocorrelation function and partial autocorrelation function. Nowadays, first the possible trend and seasonality are removed by differencing, and then p and q (usually rather small) are “guessed” checking p-values of possible models. If autoregression of residuals shows that seasonality was not completely removed, we make Ps and Qs active.

Exponential Smoothing+ARIMA

Exponential Smoothing is using past values of the time series with weights decreasing exponentially over time. In the classical form the current smoothed value is a weighted average (steered by parameter α) of the current observed value and the previous smoothed value. In our study the model has an additional parameters δ which decides the importance of past values from the seasonal period, and γ is smoothing parameter for the possible trend. Parameters are estimated tracking all values with step 0.001, and measuring the goodness-of-fit with variance of residuals.

SVR+ARIMA

Support Vector Regression fits a regression type function to the explained (dependent) variable, allowing a margin of tolerance around this function. Errors that fall within that margin don’t count. The following parameters must be estimated:

• ε (epsilon) – defines the width of the tolerance margin around the fitted function;
• C – controls how much SVR penalizes errors on points outside the tube;
• kernel – determines how SVR handles nonlinear pattern. The practical choice is between RBF (Radial Basis Function) or polynomial kernel;
• γ (gamma) – a hyperparameter that defines how far the influence of a training example reaches. It can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.

Artificial Neural Networks+ARIMA

A Multilayer Perceptron (MLP) has been chosen for our model as an artificial neural network. It is composed of fully connected layers with nonlinear activation functions. In our model the input layer receives the same features that are used in multiple regression and passes them forward. In hidden intermediate layers neurons computed sums of input, add biases, and apply nonlinear activation functions. Output layer calculates the final prediction based on the information from the final hidden layer. ReLU (rectified linear unit) was used as an activation function, and Limited-memory BFGS (L-BFGS) as optimization algorithm. Hyperparameter alpha sets the regularization power in L2, and tol defines the threshold below which, changes in objective function are so small that optimization process is terminated.

General flowchart of the proposed model is shown on Figure 1.

3. Results

The learning set in our example covers 84 days, which are 12 weeks or 2016 hours. The data is presented on Figure 2.

On Figure 2 we can clearly identify weeks and days. Typical seasonalities are present in load time series, no matter what is covered by the data. Weekly average profile of our data is presented in Figure 3.

The lowest demand for electricity was – as expected – on Sundays, so we take Sunday as base variant for dummy coding of week days. The daily profile is presented on Figure 4.

Most of the clients in the analysed area are households, what explains the highest demand for electricity between 4-11 p.m., when people are back home from offices, factories and schools. As the base for within-day hour coding we took the hour number 3, the third hour in a day.

3.1. Multiple Regression with Residual ARIMA

The basic multiple regression model for electricity load learning set is presented in Table 1. The month can be of course identified from the date, but the average daily temperature better identifies the season within the year.

Even non-significant variables marking hours have been left in the model to secure the full daily profile of the load. Residuals from the regression model are modelled (See Table 2) by ARIMA(1,0,1)(0,1,1) with seasonal period s=168.

3.2. ARIMA

Single non-seasonal differencing and single seasonal differencing with s=168 were the elements of transforming the original time series into a stationary one. ARIMA model was in the form of ARIMA(1,1,3)(1,1,1) – Table 3.

It is obvious that we did not try to estimate another ARIMA for the residuals of basic ARIMA model.

3.3. Exponential Smoothing

Automatic search has found that parameters of exponential smoothing model are equal to: alpha=1.000, delta=1.000, and gamma=0.000, which is in fact a naïve model. ARIMA model for exponential smoothing residuals is described in Table 4. Coefficients are very close to ARIMA for original series.

3.4. Support Vector Regression

In search for optimal hyperparameters of SVR, an Optuna library has been used (optuna.org). Each configuration has been verified by 10-folded cross-validation on learning set, and rated by Root Mean Square Error (RMSE). After 200 runs, the best SVR model used polynomial kernel with degree 2, and other parameters equal to:

•   C = 805.224
• epsilon = 5.083
• gamma = 0.0890
• coef0 = 1.586

3.5. Artificial Neural Network

30 NN configurations were considered in the first experiment, with one or two hidden layers, 8 to 128 neurons in the first layer, and four activation functions. The simplest networks with just one hidden layer gave the best results. So in the second experiment, there were just one layer with 2 to 16 neurons. Finally, the best network had 7 neurons, ReLU was taken as activation function. Parameters were equal to:

• alpha = 6.20 x 10^-6
• tol = 3.01 x 10^-5

3.6. Combining individual models

The combining data set consists of 48 hours prior to the forecasting period. Multiple regression model is estimated on this data. The observed electricity demand within these 48 hours is an explained variable, and theoretical values from five individual models are explaining (independent variables). This is the way how singles models are combined for the regression model which is used for the final forecast. Multiple regression model for our example is presented in Table 5.

Mean Absolute Percentage Errors for the 48 hour forecasts calculated by individual models and the final one are given in Table 6.

4. Discussion

Hourly forecasting of electricity load is important mostly from practical point of view. Companies which sale electricity rely on long-term agreements which should be adjusted on short time basis. They can buy or sale electricity in exchange platforms but the spread of prices can be substantial. Bosses of companies would like to have the forecasting error less than 1% which is almost impossible to achieve, even in normal circumstances, without strange weather events.

It is easier to forecast if the data cover more or less homogeneous region. On industrial areas, we can expect bigger drops in demand during weekends or national holidays, when factories are closed. In residential areas this effect is much smaller. We know, from our experience, that big factories (like ironworks) are usually billed separately.

Our methodological proposition is similar to bagging and boosting procedures, but there are some important differences. In classical bagging, models are trained on different bootstrap samples obtained through sampling with replacement. Our individual models are estimated using the same time series data. Boosting trains models sequentially (one after another), where each new model tries to fix the errors of the previous one. In our approach, residuals from individual models are modelled by ARIMA – of course except ARIMA model itself.

The choice and the number of individual models can always be discussed. We took models which are relatively popular in subject literature and also been used by us in other applications. Models are combined with multiple regression in which we rely not only on statistical significance but rather on goodness-of-fit. In terms of MAPE the combined model is slightly better than individual ones (Table 6). The idea is, that individual models can capture different aspects of time series variability and contribute to the combined model. The in-sample MAPE of the combined model is optimistic, since its parameters are estimated on the same 48 observations; the forecasting period gives the fair comparison.