Comparative Study of Time Series Analysis Algorithms Suitable for Short-term Forecasting in Implementing Demand Response Based on AMI

1. Introduction

Demand Response (DR) is one of the key components of the smart grid, designed to maintain the balance between electricity demand and supply and to improve the efficiency of the power grid [1]. Among the various structures of DR, the two most prominent are "Price-based Demand Response" and "Incentive-based Demand Response."

Price-based DR utilizes electricity price fluctuations to change consumers’ power consumption behaviors. It increases prices when the demand is high and decreases them when the demand is low, encouraging consumers to save electricity or adjust their consumption times. This approach includes Real-time Pricing (RTP) and Time-of-Use (TOU) pricing schemes [2].

Incentive-based DR manages the peak load of the power grid by providing economic incentives to consumers for reducing their power consumption or adjusting it to specific times. Notable examples include Demand Response Resources (DRR) and Capacity Market Payments [3,4].

Advanced Metering Infrastructure (AMI) is a fundamental component of the smart grid that significantly enhances the efficiency of Demand Response (DR) programs through real-time collection and analysis of electricity usage data [5]. AMI systems enable two-way communication between electricity consumers and utility companies, which is essential for the implementation of DR programs. Distributed DR systems play a crucial role in enhancing the resilience and reliability of the power grid, enabling rapid response times and optimizing energy management for individual users.

This study explores the applicability of various time series forecasting algorithms to enhance the efficiency of Demand Response based on Advanced Metering Infrastructure (AMI). It evaluates the capability of diverse algorithms such as ARIMA, SARIMA, LSTM, and SVM to handle the nonlinear patterns and volatility in electricity usage data. The comparative analysis of these algorithms aims to demonstrate that each has specific strengths in utilizing real-time AMI data to predict peak demand and manage loads effectively.

A key focus of this research is to strictly assess the prediction accuracy of models using real-time data, illustrating how certain algorithms can serve as powerful tools for utility companies to manage peak loads and dynamically optimize energy distribution in short-term forecasting scenarios. This analysis is expected to provide significant insights particularly for price-based Demand Response strategies like Real-time Pricing (RTP) and Time-of-Use (TOU). Additionally, it will provide evidence on how effectively incentive-based Demand Response strategies, which incentivize consumers through economic rewards to reduce power consumption, can be implemented.

The results of this study provide a foundation for future research directions and suggest the exploration of advanced machine learning techniques that can be integrated into smart grid infrastructures. Additionally, the findings offer practical implications for how algorithmic interventions can enhance the efficiency and reliability of Demand Response in establishing energy management policies.

To achieve this, this paper focuses on the selection of distributed DR methods using AMI and conducts a comparative analysis of short-term forecasting algorithms to choose the optimal demand response prediction algorithm.

2. Related work

Advanced Metering Infrastructure (AMI) represents a pivotal component of the smart grid. Its primary function encompasses tracking electricity consumption at client nodes and transmitting this information to utility centers. AMI undertakes roles in information collection, file storage, file transmission, bandwidth allocation, and billing issuance[6]. Additionally, Wang et al.[7] highlight the accuracy and reliability as significant advantages of demand response implementations in AMI systems.

• Accuracy: AMI-based demand response systems offer high precision in load control, contributing to efficient energy management and grid stability.
• Reliability: The centralized architecture and control strategies of AMI facilitate stable and trustworthy demand response operations, ensuring consistent performance.

However, smart meters within AMI are resource-constrained devices, characterized by limited computational and storage capabilities[8]. Moreover, a major challenge for AMI systems is their sustainability under unexpected usage patterns, low bandwidth, and significant informational performance degradation, with low-power-centric data technologies at the root of these issues[6]. Therefore, research in AMI functionalities must consider the constraints imposed by low-power-centric data technologies, particularly the limitations in computing and storage capabilities, which warrant serious consideration. Thus, determining the appropriate duration for data collection and analysis applied to demand response forecasting becomes crucial. While there is no standardized period for data collection and analysis for forecasting, various studies suggest differing durations: for Short Term Load Forecasting (STLF), the period ranges from less than a day to a week[9,10,11]; Medium Term Load Forecasting (MTLF) spans two weeks to three years[9,10,11]; and Long Term Load Forecasting (LTLF) encompasses three years to fifty years[9,11]. According to research by Hong et al. and Yunsun Kim et al., while no standard period exists, it is common to set STLF to less than a week, MTLF to one week to a month, and LTLF to more than a year[12,13]. Consequently, research into demand response in AMI must prioritize the selection of algorithms suitable for Short-term forecasting, considering the limitations of low-power-based technologies and the potential applicability of each algorithm to Short-term forecasting and demand response.

Recent research papers reveal that ARIMA and SARIMA models are effectively utilized in short-term forecasting, particularly in fields such as electric load forecasting and mobile network traffic predictions. These studies underscore the models’ efficacy in accurately predicting short-term variations across various sectors. For instance, research conducted by Kochetkova et al. employed SARIMA and other models for short-term forecasting of enterprise network traffic, demonstrating potential applicability in the electricity sector[14]. Minnaar et al. focused their research on using the SARIMA model for forecasting in electricity distribution, procurement, and sales, emphasizing the importance of accurate load forecasting for utility companies, especially in the context of increasing roles of renewable energy sources[15]. Moreover, studies like the one by Kien et al., which addressed SARIMA model usage for electric demand forecasting in Hanoi, showcase its effectiveness in predicting electricity demand in specific regions[16]. These examples demonstrate that ARIMA and SARIMA models are widely recognized and used across various sectors, making them highly relevant to research themes focusing on AMI and demand response.

Long Short-Term Memory (LSTM) models are deep learning models capable of effectively handling long-term dependencies in complex time series data. Developed to overcome the long-term dependency issues inherent in traditional Recurrent Neural Networks (RNNs), LSTMs have provided effective forecasting results in diverse areas, including electric load forecasting, stock market analysis, and weather condition predictions. A study by Al-Musaylh et al. (2018) that applied LSTM for electric demand forecasting illustrates the model’s capability in handling complex time series data[17]. Recent studies have explored LSTM’s approach to short-term forecasting across various fields. For example, research by Vatsa et al. employed LSTM for short-term forecasting of polarization current for transformer insulation assessment, extending LSTM architecture to capture long-term dependencies and complex temporal patterns[18].

Support Vector Machine (SVM) is another algorithm useful for short-term forecasting. Primarily known for classification problems, SVM is a robust machine learning technique that can also be applied to regression problems. It operates by mapping data features to a higher-dimensional space to find the optimal decision boundary. This characteristic makes it particularly valuable for short-term forecasting of nonlinear and complex data patterns. For instance, a study by Nepal et al. (2020) employing SVM for electric load forecasting demonstrated the model’s effectiveness in handling complex data patterns and providing accurate predictions[19].

However, existing studies tend to focus on implementing and examining a single algorithm. Hence, this paper aims to compare traditional time series analysis methods, such as ARIMA and SARIMA, with algorithms like LSTM and SVM, to enhance the efficiency of short-term forecasting in AMI systems. This comparison of diverse algorithms will play a crucial role in improving data processing and forecasting accuracy in AMI systems, ultimately contributing to better decision-making in energy management and planning.

3. Theoretical Background

3.1. ARIMA (AutoRegressive Integrated Moving Average)

The ARIMA model is a potent tool utilized for normalizing non-stationary trends in time series data and predicting future values. It combines AutoRegressive (AR) components, differencing (I) processes, and Moving Average (MA) components to analyze patterns and structures within time series data. ARIMA is applied across various fields, including economics, finance, and environmental science, showcasing exceptional performance in short-term forecasting in particular. The model identifies trends and seasonality in data over time, enabling accurate predictions[20,21].

The ARIMA model, used for analyzing time series data, is mathematically represented as follows:

$$(1-\sum _{i=1}^p{\varphi }_i{L}^i){(1-L)}^d{X}_t=(1+\sum _{i=1}^q{\theta }_i{L}^i){\epsilon }_t$$

3.2. SARIMA (Seasonal ARIMA)

SARIMA, an extension of the ARIMA model incorporating seasonal components, proves effective for data exhibiting seasonal patterns. This model integrates additional seasonal differencing and seasonal autoregressive elements into the conventional ARIMA model’s components. SARIMA is particularly beneficial for forecasting electric demand, analyzing stock markets, and researching climate change, adeptly modeling complex patterns in data with pronounced seasonal variability[22,23].

SARIMA model, an expansion of ARIMA to account for seasonal variability, is mathematically expressed as follows:

$$\begin{array}{cc}\hfill & (1-\sum _{i=1}^P{\Phi }_i{L}^{iS})(1-\sum _{i=1}^p{\varphi }_i{L}^i){(1-L)}^D{(1-L^S)}^d{X}_t\hfill \\ \hfill & =(1+\sum _{i=1}^Q{\Theta }_i{L}^{iS})(1+\sum _{i=1}^q{\theta }_i{L}^i){\epsilon }_t\hfill \end{array}$$

3.3. LSTM (Long Short-Term Memory)

LSTM, a variant of Recurrent Neural Networks (RNNs), was developed to address the issue of long-term dependencies in time series data. It can effectively learn patterns over time while maintaining important information over long periods in complex time series data. This capability has demonstrated LSTM’s potential in various fields, such as electric demand forecasting, stock market analysis, showcasing its application in areas requiring high accuracy in short-term and mid-term predictions[23,24].

The core components of LSTM and their mathematical structure are as follows:

• Forget Gate

  (a)

  Decides the extent to which information from the previous state is retained.

  $$\begin{array}{c}\hfill f_t=\sigma (W_f·[h_{t-1},x_t]+b_f)\end{array}$$

• Input Gate

  (a)

  Determines how much new information is added to the cell state.

  $$\begin{array}{c}\hfill i_t=\sigma (W_i·[h_{t-1},x_t]+b_i)\\ \hfill {\tilde{C}}_t=tanh(W_C·[h_{t-1},x_t]+b_C)\end{array}$$

• Cell State Update

  (a)

  Updates the previous cell state to create a new cell state.

  $$\begin{array}{c}\hfill C_t=f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t\end{array}$$

• Output Gate

  (a)

  Determines which parts of the cell state are used to decide the next hidden state $h_t$ for output.

  $$\begin{array}{c}\hfill o_t=\sigma (W_o·[h_{t-1},x_t]+b_o)\\ \hfill h_t=o_t\ast tanh\left(C_t\right)\end{array}$$

3.4. SVM (Support Vector Machine)

SVM is a robust machine learning algorithm widely used for solving classification and regression problems. It operates by mapping data to a higher-dimensional space and finding the optimal decision boundary. SVM is particularly effective for datasets with complex nonlinear relationships and is used in various fields, including electric demand forecasting, image classification, and bioinformatics. SVM maintains high performance even in high-dimensional data, accurately classifying and predicting complex patterns[25].

SVM classifies data by solving the following optimization problem:

$$\begin{array}{cc}\hfill \underset{\mathbf{w},b}{min}& {\displaystyle \frac{1}{2}}{\parallel \mathbf{w}\parallel }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& y_i(\mathbf{w}·{\mathbf{x}}_i+b)\ge 1,\forall i\hfill \end{array}$$

4. Research Methodology

4.1. Application of Algorithms

This section meticulously delineates the implementation methodology for each of the selected time series forecasting algorithms.

4.1.1. ARIMA (AutoRegressive Integrated Moving Average)

The ARIMA model is employed to transform non-stationary time series data into a stationary form for future value predictions. The implementation of the ARIMA model encompasses the following steps:

• Identify and preprocess time series data to remove trends and seasonality.

  (a)

  Removing Trends: Linear or nonlinear trends are eliminated from the data through differencing. The first differencing is calculated as:

  $$y_t^{\prime }=y_t-y_{t-1}$$

  where $y_t$ is the observation at time t, and $y_t^{\prime }$ is the differenced time series.

  (b)

  Removing Seasonality: Seasonal differencing is applied to eliminate seasonal variations in the data, calculated over a period S:

  $$y_t^{\prime \prime }=y_t-y_{t-S}$$

  where S represents the seasonal period.

  (c)

  Checking Stationarity: After differencing, stationarity of the time series is tested using the ADF test. If the p-value is below 0.05, the series can be considered stationary.

  (d)

  Applying Transformations: Log transformation or Box-Cox transformation is applied to stabilize the variance of the data.

  (e)

  Analyzing ACF and PACF Plots: The parameters p, d, and q of the ARIMA model are determined by analyzing the ACF and PACF plots.

• Analyze ACF and PACF plots to determine appropriate AR(p) and MA(q) parameters.

  (a)

  Analyzing ACF Plot: The ACF plot visualizes the correlation between observations in the time series data, showing the total correlation between lagged observations, and is used to estimate the order of the MA model (q). The cut-off point in the ACF plot after which correlations rapidly decrease can be chosen as the value for MA(q).

  (b)

  Analyzing PACF Plot: The PACF plot shows the correlation of individual lag values excluding the influence of other lag values, primarily used to determine the order of the AR model (p). The cut-off point in the PACF plot after which correlations rapidly decrease can be chosen as the value for AR(p).

  (c)

  Parameter Estimation: Based on ACF and PACF plots, the values of AR(p) and MA(q) best fitting the data are determined experimentally. Various combinations of p and q are tried, using information criteria (e.g., AIC, BIC) to select the optimal model.

• Apply differencing (d) to make the data stationary and fit the ARIMA model to the data.

  (a)

  Differencing: Differencing in the ARIMA model is used to transform non-stationary time series data into a stationary form.

  (b)

  Model Fitting: Fit the ARIMA model to the differenced data. The ARIMA(p, d, q) model is defined as:

  $$(1-\sum _{i=1}^p{\varphi }_i{L}^i){(1-L)}^d{y}_t=(1+\sum _{i=1}^q{\theta }_i{L}^i){\epsilon }_t$$

  (c)

  Model Estimation: Use optimization techniques to estimate the values of the model parameters $\varphi$ and $\theta$, evaluating the model’s fit and adjusting parameters as necessary.

• Validate the model’s stationarity using statistical tests like the ADF test.
• Use the final model to predict future values and evaluate the model’s performance.

  (a)

  Predicting Future Values: Use the fitted ARIMA model to predict future values. The prediction is conducted using the model’s mathematical expression.

  (b)

  Evaluating Model Performance: Analyze the difference between predicted and actual values to evaluate the model’s performance. Common metrics include RMSE or MAE for performance evaluation.

  (c)

  Model Validation: Validate the model’s fit based on prediction performance, and adjust the model as necessary. Information criteria such as AIC or BIC can be used for comparison with other models.

  Through these steps, the predictive capability of the ARIMA model is assessed, and its applicability to real-time series data is confirmed.

According to reference[23] elucidates the application of the ARIMA model to order demand forecasting, demonstrating the model’s applicability.

4.1.2. SARIMA (Seasonal ARIMA)

The SARIMA model extends the ARIMA model by incorporating seasonality, making it suitable for forecasting data with seasonal variations. The implementation of the SARIMA model includes the following steps:

• Identify and preprocess time series data including seasonal elements.

  (a)

  Identify seasonal elements and preprocess the data accordingly. This involves capturing clear seasonal patterns in the time series data and normalizing them through differencing.

  (b)

  Apply seasonal differencing to normalize the data, defined as:

  $$y_t^{\prime }=y_t-y_{t-S}$$

  where S is the seasonal period.

• Determine the seasonal parameters (P, D, Q) of the SARIMA model based on the seasonal cycle (S).
  • Decide on the seasonal parameters of the SARIMA model based on the seasonal cycle S, used to model periodic fluctuations in seasonal patterns.
  • Analyze seasonal ACF and PACF plots to determine seasonal AR and MA parameters.
• Fit the model to the data and further adjust non-seasonal parameters (p, d, q) using ACF and PACF plots.
• Evaluate the model’s fit through residual analysis and adjust the model if necessary.

  (a)

  Calculate residuals:

  $$e_t=y_t-{\widehat{y}}_t$$

  where $e_t$ are the residuals, $y_t$ the observations, and ${\widehat{y}}_t$ the predictions.

  (b)

  Assess statistical properties of residuals:

  • Verify that the mean of the residuals is close to zero.
  • Check for constant variance in the residuals (homoscedasticity).
  • Examine residuals for autocorrelation using ACF and PACF.

  (c)

  Normality test:

  • Test whether the distribution of residuals follows a normal distribution (e.g., Shapiro-Wilk test).

  Residual analysis evaluates the model’s fit and allows for adjustments to improve prediction accuracy.
• Generate forecasts considering seasonality using the final model. Predictions with the SARIMA model are made based on current and past data points and model parameters, estimating values for future points. The prediction formula is as follows:

  $${\widehat{y}}_{t+h}=\mu +\sum _{i=1}^P{\Phi }_i{y}_{t+h-iS}+\sum _{i=1}^p{\varphi }_i{y}_{t+h-i}$$

  $$-\sum _{i=1}^Q{\Theta }_i{\epsilon }_{t+h-iS}-\sum _{i=1}^q{\theta }_i{\epsilon }_{t+h-i}$$
  Where,
  • ${\widehat{y}}_{t+h}$ is the forecast value at time $t+h$.
  • $\mu$ is the mean of the model.
  • ${\Phi }_i$, ${\varphi }_i$, ${\Theta }_i$, ${\theta }_i$ are the seasonal and non-seasonal AR and MA parameters, respectively.
  • $y_{t+h-iS}$ and $y_{t+h-i}$ are past observations, and ${\epsilon }_{t+h-iS}$ and ${\epsilon }_{t+h-i}$ are past error terms.
  Using this formula, the SARIMA model predicts future values considering past data and seasonal patterns.

According to reference[24], the SARIMA model was highly effective in capturing seasonal patterns in time series data.

4.1.3. LSTM (Long Short-Term Memory)

LSTM is a deep learning-based time series prediction model capable of learning long-term dependencies in data. The implementation of the LSTM model includes the following steps:

• Normalize the time series data and prepare it by dividing sequences into windows for feeding into the network.

  (a)

  Data Normalization: The process of normalizing data to enhance the efficiency of model training. Min-Max normalization is the most common method.

  $$x_{\mathrm{normalized}}={\displaystyle \frac{x-min\left(x\right)}{max\left(x\right)-min\left(x\right)}}$$

  where x is the original data value, $min\left(x\right)$ and $max\left(x\right)$ are the minimum and maximum values of the dataset, respectively.

  (b)

  Sequence Window Splitting: Time series data is divided into continuous sequences to be learnable by the network. Each sequence consists of a specific length of windows.

  $$X_t=[x_{t-n},\dots ,x_{t-1}],Y_t=x_t$$

  where $X_t$ is the input sequence, $Y_t$ is the value to be predicted at the next time point, and n is the window size.

• Define the LSTM architecture, determining the number of layers and the number of neurons in each layer.

  (a)

  Number of Layers: The number of layers in the LSTM network determines the model’s depth, typically adjusted based on the complexity of the problem. Using more layers allows the model to learn more complex patterns but may increase the risk of overfitting.

  (b)

  Number of Neurons per Layer: The number of neurons in each LSTM layer determines the capacity of that layer. More neurons allow for storing more information but increase computational costs.

  (c)

  Architecture Example:

       model = Sequential()

       model.add(LSTM(50,

       return_sequences=True,

       input_shape=(n_input, n_features)))

       model.add(LSTM(50,

       return_sequences=False))

       model.add(Dense(1))

       model.compile(optimizer=’adam’,

       loss=’mse’)

  In this example, two LSTM layers are used with 50 neurons each. `return_sequences=True` indicates that the output of the first layer is a sequence. The number of layers and neurons per layer should be optimized through experimentation to maintain a balance between performance and efficiency of the model.

• Apply techniques to prevent overfitting using the prepared data.

  (a)

  Overfitting: Refers to a situation where the determined weights produce different results for new training data. As illustrated in Figure 1, overfitting tends to occur more frequently when there is a smaller amount of data available for training [26].

  (b)

  Measures to prevent overfitting include the following methods:

  • Cross-Validation: A method that uses a portion of the training data as validation data [24]. However, for time series data, the order of the data over time must be considered, requiring the use of Nested Cross-Validation [27] in such cases.
  • Dropout: In a fully connected neural network as shown in Figure 2, using a dropout layer as depicted in Figure 3 removes parts of the neural network for training purposes [28].

• Validate the trained model’s performance, measuring prediction accuracy using evaluation metrics.
• Perform predictions on future data using the validated model.

Reference[24] presents a case of applying LSTM for time series forecasting, demonstrating LSTM’s ability to effectively learn and predict complex data patterns.

4.1.4. SVM (Support Vector Machine)

SVM is a powerful machine learning model used for time series forecasting, particularly effective in handling nonlinear and complex data patterns. It employs a risk function based on the principle of structural risk minimization to balance empirical error and regularization.

The implementation of SVM in short-term forecasting research based on AMI involves the following steps:

• Data Preparation: Preprocess and normalize AMI data to make it suitable for SVM.
• Model Development: Configure SVM using an appropriate kernel function to capture the complexity of the data.
• Training and Testing: Train the SVM model using historical AMI data and test its performance on unseen data.
• Performance Evaluation: Assess the model’s accuracy, speed, and efficiency using metrics appropriate for time series analysis.

Through this approach, SVM can be a useful tool in AMI systems for handling complex and nonlinear data patterns.

SVM is a powerful machine learning algorithm that defines a decision boundary based on the characteristics of the dataset, finding a hyperplane with the maximum margin to classify the given data. The basic form of SVM applies to linear classification problems, but it can be extended to nonlinear classification problems using the kernel trick.

The objective function of SVM is defined as follows:

$$\underset{\mathbf{w},b}{min}{\displaystyle \frac{1}{2}}{\parallel \mathbf{w}\parallel }^2+C\sum _{i=1}^n{\xi }_i$$

Here, $\mathbf{w}$ is the weight vector of the hyperplane, b is the bias, ${\xi }_i$ are slack variables, and C is the regularization parameter. This objective function aims to find a hyperplane that maximizes the margin and minimizes classification errors.

The kernel trick allows modeling complex nonlinear relationships by mapping the input space into a higher-dimensional feature space. The most commonly used kernel function is the Radial Basis Function (RBF):

$$K({\mathbf{x}}_i,{\mathbf{x}}_j)=exp(-\gamma \parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }^2)$$

Here, $\gamma$ is the parameter of the RBF kernel, and ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ are feature vectors.

This research applies SVM to short-term electric demand forecasting, similar to methodologies proposed by Shi et al.[29] and Sun et al.[30]. These studies highlight the efficiency and accuracy of SVM, supporting the approach of this research.

5. Experiments and Results

This section introduces the experimental methods and the data used for the experiments, along with presenting the results.

5.1. Experimental Methodology

The experiments in this research will be conducted as follows:

• Collection and Trend Analysis of Time-Specific AMI Measurement Data: Collect data for training and analyze trends to understand the actual movements within the AMI measurement data.

  (a)

  Data collection for training

  (b)

  Data collection for comparison

• Applying Collected Data to Each Algorithm: The collected data is applied to each algorithm.
• Trend Comparison: The trends from the results of applying each algorithm are compared with previously observed trends to analyze differences.
• Data Comparison: Directly compare the data collected for comparison with the results from each algorithm to analyze the differences.

Through the above processes, the performance of each algorithm will be indirectly assessed to determine which algorithm performs best. Additionally, considering the operation based on AMI, the experiments will be conducted in a virtual environment configured with a 1Ghz 2core CPU and 800MB RAM.

5.2. Introduction to Experimental Data

KT, the National Information Society Agency (NIA), and the Ministry of Science and ICT of South Korea provide big data across various industrial sectors (https://www.bigdata-telecom.kr/). The big data used in this experiment are as follows:

• Data for comparison: Time-specific AMI AI training data

  (a)

  Data period: December 1, 2021, to January 1, 2022

  (b)

  Data size: 5,426,322,257 Bytes

• Data for training: Time-specific AMI electricity usage

  (a)

  Data period: July 1, 2021, to December 1, 2021

  (b)

  Data size: 15,479,044,282 Bytes

The data collected for final comparison is named "Time-specific AMI AI training data," and the data collected for training is named "Time-specific AMI electricity usage."

Table 1 defines the "Time-specific AMI AI training data" file, intending to verify the data and its trends based on the Consumer Number $(CNSMR\_NO)$, Measurement Date $(MESURE\_DE)$, and Measurement Time $(MESURE\_TM)$.

5.3. Introduction to the Experimental Method for Learning AMI Time-Specific Power Usage Data

The collected "AMI Time-Specific Power Usage Data," like the previously collected "AMI Time-Specific AI Learning Data" for comparison, is provided in csv format, representing data measured in apartments equipped with actual AMI devices, organized by time of day. The data from July 1, 2021, to December 1, 2021, will be used for training each algorithm, and the results will be saved in csv files in the same format as the "AMI Time-Specific AI Learning Data" for direct comparison in the previous subsection "Collection and Trend Analysis of Time-Specific AMI Measurement Data." To facilitate this, data files from July 1, 2021, to December 1, 2021, were merged into one file, resulting in a csv file with a total size of 15,479,044,282 bytes and containing 316,712,453 lines. From this file, data for the 10 consumer numbers $(CNSMR\_NO)$ sampled in Figure 4 will be extracted. The definition of this recreated csv format can be found in Table 2.

The code to be implemented for the experiment and the hardware specifications on which the experiment will be conducted are as follows:

• Language: Python 3.11.7
• Used Libraries

  (a)

  pandas

  (b)

  pmdarima

  (c)

  numpy

  (d)

  tqdm

  (e)

  datetime

  (f)

  sys

  (g)

  sklearn

  (h)

  tensorflow

• Virtual Environment Hardware Specifications $\left(virtualmachine\right)$

  (a)

  CPU: 1.0GHz 2core

  (b)

  RAM: 800MB

5.4. ARIMA Algorithm Experiment and Results

Algorithm 1 presents the actual implementation, encompassing the majority of the steps involved in the implementation process of the proposed ARIMA model.

Algorithm 1 Forecasting and Saving Power Usage Data Using ARIMA

1: functionPreprocessChunk($chunk$)    2:     Combine ’MESURE_YEAR’, ’MESURE_MT’, ’MESURE_DTTM’ into ’Date’    3:     return $chunk$    4: end function    5: functionLoadAndPreprocessData($filepath$, $chunksize$)    6:     Load data in chunks from $filepath$ using specified columns    7:     Apply PreprocessChunk to each chunk    8:     Concatenate all processed chunks into a single DataFrame    9:     return the concatenated DataFrame  10: end function  11: functionAutoSelectOrderAndPredict($df$, $n\_periods$)  12:     Initialize an empty list for results  13:     for each group in $df$ grouped by ’LEGALDONG_CD’ and ’CNSMR_NO’ do  14:         Extract ’SGPowerUsage’ series from the group  15:         Fit auto ARIMA model and predict for $n\_periods$  16:         Append forecast results to the results list  17:     end for  18:     return list of forecast results  19: end function  20: functionSaveForecast($results$, $output\_file$, $start\_of\_next\_month$, $n\_periods$)  21:     Generate date and hour range for the next month starting from  $start\_of\_next\_month$  22:     Convert $results$ to a DataFrame with proper date and hour columns  23:     Save the DataFrame to $output\_file$  24: end function  25: procedureMain($input\_csv\_file$, $output\_csv\_file$)  26:     $df\leftarrow$ Load and preprocess data from $input\_csv\_file$  27:     Calculate the start of next month and the number of periods ($n\_periods$)  28:     $results\leftarrow$ Perform ARIMA forecasting for each group in $df$  29:     Save forecast results to $output\_csv\_file$  30: end procedure

Table 3 summarizes the slope $\left(slope\right)$ and intercept $\left(intercept\right)$ of the calculated trend lines for each consumer number $(CNSMR\_NO)$.

5.5. SARIMA Algorithm Experiment and Results

Algorithm 2 presents the actual implementation, encompassing the majority of the steps involved in the implementation process of the proposed SARIMA model.

Algorithm 2 Forecasting Power Usage with Seasonality and Saving the Results

1: functionPreprocessChunk($chunk$)    2:     Combine ’MESURE_YEAR’, ’MESURE_MT’, ’MESURE_DTTM’, ’Hour’ into ’DateTime’    3:     Create a dummy variable ’IsHoliday’ from ’HLDY_AT’    4:     return $chunk$    5: end function    6: functionLoadAndPreprocessData($filepath$, $chunksize$)    7:     Load data in chunks from $filepath$ using specified columns    8:     Apply PreprocessChunk to each chunk    9:     Concatenate all processed chunks into a single DataFrame  10:     return the concatenated DataFrame  11: end function  12: functionAutoSelectOrderAndPredict($df$, $n\_periods$)  13:     Initialize an empty list for results  14:     for each group in $df$ grouped by ’LEGALDONG_CD’ and ’CNSMR_NO’ do  15:         Prepare the time series and holiday series sorted by ’DateTime’  16:         Fit SARIMA model with holiday information as an exogenous variable  17:         Forecast $n\_periods$ ahead with the last week’s holiday data  18:         Append forecast results and dates to the results list  19:     end for  20:     return list of forecast results  21: end function  22: functionSaveForecast($results$, $output\_file$)  23:     Convert $results$ to a DataFrame with date and hour separated  24:     Save the DataFrame to $output\_file$  25: end function  26: procedureMain($input\_csv\_file$, $output\_csv\_file$)  27:     $df\leftarrow$ Load and preprocess data from $input\_csv\_file$  28:     Calculate the start of next month and determine $n\_periods$ for forecasting  29:     $results\leftarrow$ Perform SARIMA forecasting for each group in $df$  30:     Save forecast results to $output\_csv\_file$  31: end procedure

Table 4 summarizes the slope $\left(slope\right)$ and intercept $\left(intercept\right)$ of the calculated trend lines for each consumer number $(CNSMR\_NO)$.

5.6. LSTM Algorithm Experiment and Results

Algorithm 3 presents the actual implementation, encompassing the majority of the steps involved in the implementation process of the proposed LSTM model.

Algorithm 3 Forecasting Power Usage with LSTM Model for Each Consumer Group

1: functionPreprocessData($df$)    2:     Normalize ’SGPowerUsage’ using MinMaxScaler    3:     return $df$, $scaler$    4: end function    5: functionCreateSequences($df$, $n\_steps$)    6:     Initialize empty arrays for X (features) and y (labels)    7:     for each i from $n\_steps$ to $len\left(df\right)$ do    8:         Append scaled usage values from $i-n\_steps$ to i to X    9:         Append scaled usage value at i to y  10:     end for  11:     return X, y  12: end function  13: functionBuildLSTMModel($input\_shape$)  14:     Create Sequential model with LSTM and Dense layers (units=50)  15:     Compile model with ’adam’ optimizer and mean squared error loss  16:     return $model$  17: end function  18: functionPredictForEachGroup($group$, $n\_steps$, $model$, $scaler$)  19:     Initialize list for predictions  20:     Prepare input sequence for the next month’s first prediction  21:     for each hour in next month do  22:         Predict next hour’s usage and inverse transform the scaled value  23:         Append prediction to the list  24:         Update input sequence with the new prediction  25:     end for  26:     return predictions  27: end function  28: procedureMain($input\_csv\_file$, $output\_csv\_file$, $n\_steps$)  29:     Load and preprocess data from $input\_csv\_file$  30:     Group data by ’LEGALDONG_CD’ and ’CNSMR_NO’  31:     Initialize list for all predictions  32:     for each group do  33:         Sort group by date and reset index  34:         if group size ≥$n\_steps$ then  35:            X, y←CreateSequences(group, $n\_steps$)  36:            $model$←BuildLSTMModel(($n\_steps$, 1))  37:            Train $model$ with early stopping  38:            $predictions$←PredictForEachGroup(group, $n\_steps$, $model$, $scaler$)  39:            Append $predictions$ to the all predictions list  40:         end if  41:     end for  42:     Save all predictions to $output\_csv\_file$  43: end procedure

Table 5 summarizes the slope $\left(slope\right)$ and intercept $\left(intercept\right)$ of the calculated trend lines for each consumer number $(CNSMR\_NO)$.

5.7. SVM (Support Vector Machine) Algorithm Experiment and Results

Algorithm 4 presents the actual implementation, encompassing the majority of the steps involved in the implementation process of the proposed SVM model.

Algorithm 4 Predict Future Power Usage Using SVR Model

1: functionPreprocessData($filepath$)    2:     Load data from $filepath$ into DataFrame $df$    3:     Convert date columns to ’Date’ in $yyyymmdd$ format    4:     Ensure ’Hour’ is integer    5:     return $df$    6: end function    7: functionGenerateFutureDates($last\_date\_str$, $n\_days$)    8:     Calculate start of next month from $last\_date\_str$    9:     Generate hourly dates for $n\_days$ into the future  10:     return list of future dates  11: end function  12: functionPrepareFutureDataset($df$, $future\_dates$)  13:     Create empty DataFrame for future data  14:     for each $date$ in $future\_dates$ do  15:         Create temporary DataFrame for the date  16:         Append to future data DataFrame  17:     end for  18:     return DataFrame for future predictions  19: end function  20: functionTrainAndPredict($df$)  21:     Initialize list for predictions  22:     for each group in $df$ by ’CNSMR_NO’ and ’LEGALDONG_CD’ do  23:         Separate features and target  24:         Scale features using StandardScaler  25:         Split data into training and test sets  26:         Train SVR model on training set  27:         Prepare dataset for future predictions  28:         Scale future features and predict future power usage  29:         Append predictions to list  30:     end for 31:     Combine all predictions into a DataFrame  32:     return DataFrame with predictions  33: end function  34: procedureMain($input\_csv\_file$, $output\_csv\_file$)  35:     $df\leftarrow$PreprocessData($input\_csv\_file$)  36:     $predictions\_df\leftarrow$TrainAndPredict($df$)  37:     Save $predictions\_df$ to $output\_csv\_file$  38: end procedure

Table 6 summarizes the slope $\left(slope\right)$ and intercept $\left(intercept\right)$ of the calculated trend lines for each consumer number $(CNSMR\_NO)$.

5.8. Experimental Results and Comparison

From Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14, graphs feature a red zero line. This line indicates that the closer the predicted data, represented in blue, is to zero, the closer it is to the actual data. Furthermore, the predicted results for each consumer number $(CNSMR\_NO)$ will be quantitatively evaluated using three evaluation methods: MSE, MAE, and RMSE.

• Mean Squared Error $\left(MSE\right)$

  (a)

  The differences between actual and predicted values are squared, averaged, and then calculated.

  (b)

  $$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(Y_i-{\widehat{Y}}_i)}^2$$

  (c)

  Here, $Y_i$ represents the actual value, ${\widehat{Y}}_i$ the predicted value, and n the number of samples. Since MSE squares the errors, it gives more weight to larger errors, making it a useful tool for emphasizing model performance in situations where larger errors are disadvantageous. Therefore, it has the characteristic of being more sensitive to larger errors [31].

• Mean Absolute Error $\left(MAE\right)$

  (a)

  Defined as the average of the absolute differences between the predicted values and the actual values.

  (b)

  $$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n|Y_i-{\widehat{Y}}_i|$$

  (c)

  Unlike MSE, MAE evaluates model errors linearly, making it less sensitive to larger errors. Therefore, this characteristic makes it a metric that can be used when all errors need to be treated uniformly. It differentiates itself from MSE by treating all errors equally [31].

• Root Mean Squared Error $\left(RMSE\right)$

  (a)

  By taking the square root of MSE, the utility of MSE can be further extended, thereby restoring the error metric to the same scale as the predicted data.

  (b)

  $$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{(Y_i-{\widehat{Y}}_i)}^2}$$

  (c)

  RMSE combines the sensitivity of MSE to large errors with the advantage of being in the same units as the response variable, making it an essential metric for comparing and evaluating various regression models on a consistent scale [31].

In the domain of predictive modeling and data analysis, the Mean Squared Error (MSE), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) serve as fundamental metrics for evaluating the performance and accuracy of regression models. Each metric offers a distinctive approach to quantifying and analyzing the discrepancies between the model’s predicted values and the actual observed data, thereby enabling a robust understanding of the model’s effectiveness in relation to empirical data. Algorithm 5 represents the implementation code for each analysis method.

Algorithm 5 Calculate Error Metrics per CNSMR_NO

1: procedureLoadData($predicted\_path$, $actual\_path$)    2:     $predicted\_data\leftarrow$ Load CSV from $predicted\_path$ without headers    3:     $actual\_data\leftarrow$ Load CSV from $actual\_path$ without headers    4:     Set column names for both datasets    5:     return $predicted\_data$, $actual\_data$    6: end procedure    7: procedureMergeData($predicted\_data$, $actual\_data$)    8:     Merge on ’CNSMR_NO’, ’Date’, ’Hour’ with suffixes    9:     return $merged\_data$  10: end procedure  11: functionCalculateMetrics($group$)  12:     $error\leftarrow$ Calculate difference between predictions and actuals  13:     $mse\leftarrow$ Calculate mean squared error  14:     $mae\leftarrow$ Calculate mean absolute error  15:     $rmse\leftarrow$ Calculate root mean squared error  16:     return $mse$, $mae$, $rmse$  17: end function  18: procedureMain($predicted\_csv$, $actual\_csv$)  19:     $predicted\_data$, $actual\_data$←LoadData($predicted\_csv$, $actual\_csv$)  20:     $merged\_data\leftarrow$MergeData($predicted\_data$, $actual\_data$)  21:     Initialize $error\_metrics$ as an empty dictionary  22:     for each $consumer\_no$ in $merged\_data$ grouped by ’CNSMR_NO’ do  23:         $mse$, $mae$, $rmse$← CalculateMetrics(group)  24:         Store $mse$, $mae$, $rmse$ in $error\_metrics$ with key $consumer\_no$  25:     end for  26:     Print $error\_metrics$  27: end procedure

Figures illustrating the differences between predicted and actual data, and Algorithm 5 employing each algorithm’s predicted and actual data to assess accuracy using the MSE, MAE, and RMSE methods, are followed by tables summarizing the evaluation metrics. The overall analysis based on these evaluation metrics yields the following results.

• Consumer Number (CNSMR_NO): DJ0200309001501

  (a)

  Figure 5: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 7: Accuracy evaluation for the predicted data using MSE, MAE, RMSE

  (c)

  Algorithm-specific evaluation results according to Table 7

  • ARIMA: With results of MSE $\left(0.144\right)$, MAE $\left(0.253\right)$, RMSE $\left(0.380\right)$, demonstrating performance comparable to SARIMA.
  • SARIMA: Exhibiting MSE $\left(0.143\right)$, MAE $\left(0.250\right)$, RMSE $\left(0.378\right)$, indicating performance second only to SVM.
  • LSTM: Showing the highest error values with MSE $\left(0.675\right)$, MAE $\left(0.777\right)$, RMSE $\left(0.821\right)$, indicating the lowest prediction accuracy for this dataset.
  • SVM: Achieving the lowest MSE $\left(0.076\right)$, MAE $\left(0.198\right)$, RMSE $\left(0.276\right)$, signifying the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): DJ0800133001204

  (a)

  Figure 6: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 8: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 8

  • ARIMA: With MSE $\left(0.018\right)$, MAE $\left(0.094\right)$, and RMSE $\left(0.134\right)$, showing similar performance to SARIMA.
  • SARIMA: Shows slightly higher error rates than ARIMA with MSE $\left(0.019\right)$, MAE $\left(0.097\right)$, and RMSE $\left(0.138\right)$, positioning it as second-best after SVM.
  • LSTM: Recorded the highest error rates with MSE $\left(0.065\right)$, MAE $\left(0.232\right)$, and RMSE $\left(0.255\right)$, indicating the lowest prediction accuracy for this dataset.
  • SVM: Exhibited the lowest error rates with MSE $\left(0.009\right)$, MAE $\left(0.068\right)$, and RMSE $\left(0.097\right)$, signifying the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): DJ1200215000404

  (a)

  Figure 7: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 9: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 9

  • ARIMA: Demonstrates similar performance to SARIMA with MSE $\left(0.024\right)$, MAE $\left(0.124\right)$, and RMSE $\left(0.154\right)$, indicating closely matching results down to several decimal places.
  • SARIMA: Exhibits comparable results to ARIMA, positioning it as effective after SVM in terms of performance.
  • LSTM: Shows the highest error rates with MSE $\left(0.057\right)$, MAE $\left(0.208\right)$, and RMSE $\left(0.238\right)$, indicating the lowest prediction accuracy for this dataset.
  • SVM: Achieves the lowest error rates with MSE $\left(0.009\right)$, MAE $\left(0.071\right)$, and RMSE $\left(0.093\right)$, signifying the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): CB0100106000505

  (a)

  Figure 8: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 10: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 10

  • ARIMA: Recorded the highest error with MSE $\left(0.034\right)$, MAE $\left(0.133\right)$, and RMSE $\left(0.185\right)$.
  • SARIMA: Demonstrates slightly better performance than ARIMA with MSE $\left(0.032\right)$, MAE $\left(0.130\right)$, and RMSE $\left(0.179\right)$, making it second-best after SVM.
  • LSTM: Shows marginally higher error than SARIMA with MSE $\left(0.034\right)$, MAE $\left(0.132\right)$, and RMSE $\left(0.184\right)$ for this dataset.
  • SVM: Exhibited the lowest error rates with MSE $\left(0.029\right)$, MAE $\left(0.117\right)$, and RMSE $\left(0.169\right)$, indicating the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): CN0100107001801

  (a)

  Figure 9: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 11: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 11

  • ARIMA: Exhibited MSE $\left(0.026\right)$, MAE $\left(0.132\right)$, and RMSE $\left(0.160\right)$, showing very similar performance to SARIMA.
  • SARIMA: Recorded MSE $\left(0.025\right)$, MAE $\left(0.132\right)$, and RMSE $\left(0.160\right)$, providing the most accurate predictions for this dataset.
  • LSTM: Demonstrated the highest error rates with MSE $\left(0.065\right)$, MAE $\left(0.216\right)$, and RMSE $\left(0.254\right)$ for this dataset.
  • SVM: Showed slightly higher error rates with MSE $\left(0.026\right)$, MAE $\left(0.132\right)$, and RMSE $\left(0.161\right)$ compared to ARIMA and SARIMA, but remained close in prediction accuracy.

• Consumer Number (CNSMR_NO): CN0200311001801

  (a)

  Figure 10: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 12: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 12

  • ARIMA: Displayed MSE $\left(0.111\right)$, MAE $\left(0.178\right)$, and RMSE $\left(0.333\right)$, showing nearly identical performance to SARIMA.
  • SARIMA: Also recorded MSE $\left(0.111\right)$, MAE $\left(0.178\right)$, and RMSE $\left(0.333\right)$, demonstrating performance second only to SVM.
  • LSTM: Had the highest error rates with MSE $\left(0.117\right)$, MAE $\left(0.185\right)$, and RMSE $\left(0.342\right)$ for this dataset.
  • SVM: Achieved the lowest error rates with MSE $\left(0.087\right)$, MAE $\left(0.161\right)$, and RMSE $\left(0.295\right)$, indicating the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): CN1100106000103

  (a)

  Figure 11: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 13: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 13

  • ARIMA: Demonstrated MSE $\left(0.136\right)$, MAE $\left(0.242\right)$, RMSE $\left(0.368\right)$.
  • SARIMA: Showed identical performance to ARIMA with MSE $\left(0.136\right)$, MAE $\left(0.242\right)$, RMSE $\left(0.368\right)$.
  • LSTM: Recorded the highest error rates with MSE $\left(0.156\right)$, MAE $\left(0.255\right)$, RMSE $\left(0.395\right)$ in this dataset.
  • SVM: Achieved the lowest error rates with MSE $\left(0.096\right)$, MAE $\left(0.187\right)$, RMSE $\left(0.310\right)$, providing the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): CN1600102001004

  (a)

  Figure 12: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 14: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 14

  • ARIMA: Reported MSE $\left(0.026\right)$, MAE $\left(0.122\right)$, and RMSE $\left(0.162\right)$.
  • SARIMA: Exhibited identical performance to ARIMA with MSE $\left(0.026\right)$, MAE $\left(0.122\right)$, and RMSE $\left(0.162\right)$.
  • LSTM: Recorded the highest error rates with MSE $\left(0.063\right)$, MAE $\left(0.216\right)$, and RMSE $\left(0.251\right)$ for this dataset.
  • SVM: Achieved the lowest error rates with MSE $\left(0.011\right)$, MAE $\left(0.082\right)$, and RMSE $\left(0.106\right)$, providing the most accurate predictions for this dataset.

• Consumer Number (CNSMR_NO): CN0500311000403

  (a)

  Figure 13: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 15: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 15

  • ARIMA: Exhibited MSE $\left(0.057\right)$, MAE $\left(0.190\right)$, and RMSE $\left(0.232\right)$, indicating the highest error values for this dataset.
  • SARIMA: Reported MSE $\left(0.054\right)$, MAE $\left(0.190\right)$, and RMSE $\left(0.232\right)$, showing slightly higher errors compared to SVM.
  • LSTM: Achieved the lowest error rates with MSE $\left(0.024\right)$, MAE $\left(0.122\right)$, and RMSE $\left(0.156\right)$, providing the most accurate predictions for this dataset.
  • SVM: Recorded MSE $\left(0.034\right)$, MAE $\left(0.147\right)$, and RMSE $\left(0.184\right)$, demonstrating the second-best performance after LSTM.

• Consumer Number (CNSMR_NO): CN0700109000102

  (a)

  Figure 14: Graphs illustrating the differences between algorithm-specific predicted and actual data

  (b)

  Table 16: Accuracy evaluation for the predicted data using MSE, MAE, RMSEfor each algorithm

  (c)

  Evaluation results for each algorithm based on Table 16

  • ARIMA: Achieved MSE $\left(0.014\right)$, MAE $\left(0.085\right)$, RMSE $\left(0.120\right)$, providing the most accurate predictions for this dataset.
  • SARIMA: Recorded identical performance to ARIMA with MSE $\left(0.014\right)$, MAE $\left(0.085\right)$, RMSE $\left(0.120\right)$.
  • LSTM: Showed slightly higher error rates with MSE $\left(0.017\right)$, MAE $\left(0.095\right)$, RMSE $\left(0.130\right)$ compared to ARIMA and SARIMA.
  • SVM: Reported the highest error rates with MSE $\left(0.026\right)$, MAE $\left(0.124\right)$, RMSE $\left(0.161\right)$ for this dataset.

Figure 15 illustrates the overall differences in SGPowerUsage values between the predictions and actual data for each model $(SVM,LSTM,SARIMA,ARIMA)$. For reference, each graph includes a red dotted zero line. Points above this line indicate an overestimation by the model, while points below indicate an underestimation.

• ARIMA: Exhibits performance similar to SARIMA but with a slightly higher MAE.
• SARIMA: Generally closer to the zero line compared to LSTM, though with some deviations, which are reflected in the MAE.
• LSTM: Shows larger deviations from the zero line, resulting in a higher MAE compared to other models.
• SVM: Demonstrates relatively small fluctuations around the zero line, indicating predictions closer to the actual values, consistent with the lowest MAE.

Based on the overall experimental results, it can be concluded that SVM is the most accurate algorithm for scenarios like short-term forecasting in AMI, where the dataset is small or limited resources are available for prediction.

6. Discussion

In this study, the performance of four predictive models, SVM, ARIMA, SARIMA, and LSTM, was evaluated across various time series datasets, aiming to improve the accuracy of predicting actual electricity usage $\left(SGPowerUsage\right)$. The results were derived using statistical metrics such as Mean Squared Error $\left(MSE\right)$, Mean Absolute Error $\left(MAE\right)$, and Root Mean Squared Error $\left(RMSE\right)$.

Initially, the expected performance ranking of the algorithms "ARIMA", "SARIMA", "LSTM", "SVM", and the rationale were as follows:

• LSTM: Given its capability to learn long-term dependencies in time series data and model complex patterns and nonlinearity, LSTM was anticipated to be highly effective, especially with large datasets containing more information and patterns. Additionally, LSTM can efficiently process large datasets using GPU acceleration, making it feasible for learning from substantial data given adequate computing resources [32,33].
• SVM: Useful for classifying or regressing nonlinear patterns in high-dimensional spaces through kernel tricks, SVM might be slower than LSTM for large datasets but can effectively model specific nonlinear patterns. Its flexibility with various kernel functions allows for adaptive transformation of data features, capturing the characteristics of time series data more accurately [34,35].
• SARIMA: Specialized in analyzing time series data with seasonality, SARIMA could effectively model this aspect in large datasets with clear seasonal patterns. It is interpretable and allows for a clear understanding of the seasonal patterns and trends in the data through its modeling results [36].
• ARIMA: Strong in modeling non-seasonal linear time series data, ARIMA could be useful for datasets primarily exhibiting linear patterns, although it might be less suitable for datasets with seasonality or nonlinear patterns. Despite potential limitations in processing large datasets, ARIMA’s relatively simple computations for model construction and prediction could be advantageous in scenarios with limited computing resources [37].
• Conclusion: Understanding the characteristics and requirements of the data accurately and selecting the appropriate model is crucial. While LSTM might be most promising for large datasets, SARIMA or SVM could be considered for clear modeling of seasonality or specific patterns. ARIMA could be more suitable for simpler time series data.

However, the outcomes of this research showed that the Support Vector Machine $\left(SVM\right)$ model exhibited the most superior prediction performance across most datasets by recording the lowest error metrics. This can be attributed to SVM’s excellent capability in handling complex nonlinear patterns and its superior generalization ability. In particular, SVM leveraged the kernel trick to effectively process data characteristics in high-dimensional spaces, demonstrating exceptional performance in capturing the complex structures and dynamics within time series data [38].

Meanwhile, Autoregressive Integrated Moving Average $\left(ARIMA\right)$ and Seasonal Autoregressive Integrated Moving-Average $\left(SARIMA\right)$ models exhibited strengths in modeling the seasonality and trends of time series data but showed relatively higher error metrics compared to SVM. This indicates that while ARIMA and SARIMA models can effectively reflect linear trends in time series data, they have limitations in modeling nonlinear patterns and complex interactions within the data [39].

The Long Short-Term Memory $\left(LSTM\right)$ model demonstrated outstanding prediction performance in certain datasets but did not consistently exhibit good performance. The performance of LSTM significantly depends on having sufficient training data and proper hyperparameter settings [40,41,42], suggesting that these conditions might not have been optimized in this study.

Moreover, when comparing "ARIMA", "SARIMA", "LSTM", "SVM" algorithms directly in scenarios with limited resources and small datasets for training, such as AMI, it’s crucial to consider various aspects of performance and efficiency, including data processing capabilities, model complexity, training time, prediction performance, and memory management.

• ARIMA and SARIMA: As traditional time series analysis methods, they can be relatively fast and efficient in processing large datasets. However, their computational complexity may increase with the size of the time units in the data, and estimating the model’s parameters in large datasets can be time-consuming.
• LSTM: Possesses a strong capability to learn long-term dependencies and complex patterns in very large datasets. Training LSTM on very large datasets, however, can be computationally expensive and time-consuming.
• SVM: While effective at handling nonlinear patterns, learning times and memory requirements can significantly increase for large datasets.

7. Conclusion

This study aimed to evaluate the performance of various time series prediction models and conduct an in-depth analysis of each model’s strengths and weaknesses to contribute to selecting appropriate predictive models for real-world problems like electricity usage forecasting in AMI. Based on the research findings, choosing the right model based on the characteristics of the data and the prediction goals is crucial. Moreover, to improve the prediction accuracy for complex time series data, optimizing the model’s hyperparameters and considering the application of ensemble techniques combining various models in AMI should be explored.

Based on the results of this study, the combinations expected to enhance performance and accuracy in AMI when algorithms are combined are as follows:

• SARIMA + SVM combination or ARIMA + SVM combination: SARIMA or ARIMA can handle the linear or seasonal patterns in time series data, while SVM can complement nonlinear patterns. Especially as observed in this study, SVM can be useful for analyzing nonlinear patterns in small datasets or specific time segments. A strategy of training and predicting models targeting specific segments of the data can be employed.
• LSTM + SARIMA combination: LSTM has excellent capabilities in learning nonlinear patterns and long-term dependencies, while SARIMA can model clear patterns like seasonality. LSTM can be used to learn complex patterns, and SARIMA to complement seasonal patterns. However, this combination may require very high resource usage.

For short-term forecasting of demand response in AMI, both the model’s performance and efficiency must be considered. LSTM has strengths in learning complex nonlinear patterns but requires high resources for learning on large datasets. Conversely, ARIMA and SARIMA have relatively low computational costs but limited capabilities in learning nonlinear patterns. SVM can be useful for nonlinear patterns. Therefore, selecting or combining the appropriate model(s) based on the characteristics of the data and the requirements of the prediction task is crucial.

Future studies should further investigate the SARIMA + SVM combination or the ARIMA + SVM combination and the LSTM + SARIMA combination to analyze the accuracy achieved with these combinations and compare the resource usage related to accuracy improvements. This will help more accurately select the most suitable methodology for short-term forecasting of demand response in AMI. Additionally, while this research utilized datasets in a simulated environment rather than actual AMI, subsequent studies should aim to derive more in-depth and specific results using limited resources in actual AMI environments. This will contribute to selecting appropriate predictive models for real-world problems like electricity usage forecasting in AMI.