An Ensemble Approach to Stock Price Prediction Using Deep Learning and Time Series Models

1. Introduction

Predicting stock prices accurately is a formidable challenge due to the inherent volatility and complexity of financial markets. Retail investors, in particular, face significant hurdles as they often lack access to advanced tools and the expertise necessary for sophisticated quantitative trading. Traditional methods such as fundamental and technical analysis have been the mainstay for these investors. However, the advent of machine learning has introduced new possibilities, promising enhanced accuracy and reliability in stock price prediction.

The JPX Tokyo Stock Exchange prediction project aims to leverage advanced machine learning techniques to assist retail investors in making better-informed trading decisions. This research utilizes data from January 4, 2017, to December 3, 2021, providing a comprehensive dataset for developing and testing predictive models. The goal is to create an ensemble model that integrates various machine learning algorithms to improve prediction performance.

Data preprocessing is crucial, especially with financial data that can be noisy and incomplete. Techniques such as mean imputation for missing values and the interquartile range method for outliers ensure the dataset is clean and reliable. Feature engineering involves creating new features to enhance model performance, such as technical indicators like moving averages, relative strength index (RSI), and Bollinger Bands, which capture important market trends.

We employ various algorithms, including Keras Deep Neural Networks (DNN), LightGBM, Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), and linear regression (LR). DNNs are known for learning complex patterns from large datasets. Using the Keras library, we design DNN models for stock price prediction, experimenting with different architectures and hyperparameters. LightGBM, a gradient boosting framework, is highly effective for high-dimensional and complex data. By building an ensemble of weak learners, typically decision trees, LightGBM enhances predictive accuracy.

RNNs, particularly LSTM and GRU, excel at modeling time series data due to their ability to capture temporal dependencies. These models learn long-term dependencies and sequential patterns, essential for predicting future stock prices. Linear Regression (LR) serves as a fundamental technique, providing a baseline for comparison.

The final stage integrates these models into an ensemble. Ensemble learning improves overall performance by combining multiple models, leveraging their strengths and mitigating weaknesses. Techniques such as stacking, bagging, and boosting develop a robust predictive model. Stacking involves training multiple base models and using their predictions as inputs to a meta-model, typically a simple linear model. Bagging, or bootstrap aggregating, trains multiple models on different data subsets and averages their predictions. Boosting builds models sequentially, with each model correcting the errors of its predecessor.

This research’s contributions include a comprehensive data preprocessing pipeline, extensive feature engineering, multiple machine learning models, and an effective ensemble. Through rigorous preprocessing, feature engineering, and model optimization, we aim to create a robust predictive model surpassing existing approaches. Our findings demonstrate the practical applications of combining different machine learning techniques for retail investors. Future work will refine the model and explore additional features and techniques to enhance performance further.

2. Related Work

In the field of stock price prediction, a variety of machine learning techniques have been explored, each offering distinct advantages and facing specific limitations. Traditional methods such as linear regression (LR) have been widely adopted due to their simplicity and interpretability. However, these methods often struggle to capture the complex, non-linear relationships inherent in stock market data. More advanced approaches are needed to improve predictive accuracy[1].

Artificial Neural Networks (ANNs) represent one such advanced approach, capable of learning from large datasets to identify intricate patterns. Rather et al.[2] demonstrated the potential of ANNs in forecasting stock prices, showcasing their ability to handle non-linear relationships. Despite their strengths, ANNs require significant computational resources and are prone to overfitting, particularly when applied to noisy financial data.

Xu et al.[3] introduce a method for preprocessing data that effectively handles missing values and outliers, which is crucial in our methodology for ensuring clean input data for stock price prediction.Smith et al.[4] provide insights into feature engineering techniques, which we applied to enhance the predictive power of our machine learning models by creating more informative features.Johnson et al.[5] emphasize the importance of cross-validation in model training, which we used to ensure the robustness and generalizability of our stock price prediction models.Gupta et al.[6] discuss parameter tuning for machine learning models, guiding our approach to optimize the performance of Keras DNN, LightGBM, LSTM, GRU, and LR models.

Liu et al.[7] highlight the advantages of using ensemble models for improved prediction accuracy, which inspired the integration of time series and deep learning models in our study.Chen et al.[8] demonstrate the application of deep learning techniques, such as LSTM and GRU, in financial data analysis, which we incorporated to capture temporal dependencies in stock price data.Zhou et al.[9] discuss the application of LightGBM in handling large datasets efficiently, which we employed to manage the extensive data from the stock exchange market.Wang et al. [10] provide a framework for combining different machine learning models to enhance prediction accuracy, supporting our approach to use an ensemble model.Li et al.[11] explore the application of neural networks in financial predictions, which informed our use of Keras DNN to model complex patterns in stock price movements.

Recurrent Neural Networks (RNNs), particularly Long Short-Term Memory (LSTM) networks, have gained prominence for their ability to model time-dependent data. LSTMs are designed to capture temporal dependencies, making them well-suited for time series forecasting. Brownlee[12] highlighted the effectiveness of LSTMs in various applications, including stock price prediction. Fischer and Krauss[13] successfully applied LSTMs to stock price prediction, demonstrating their capability to capture sequential patterns. However, LSTMs can be computationally intensive and challenging to optimize.

Gradient Boosting Machines, such as LightGBM, offer another promising approach by combining the predictions of multiple weak learners to enhance overall performance. LightGBM has been praised for its efficiency and scalability, making it suitable for large and complex datasets. Ke et al.[14] demonstrated the efficacy of LightGBM in various predictive tasks, including financial predictions. The boosting technique iteratively improves the model by focusing on difficult cases, leading to significant performance gains.

In recent years, ensemble methods that integrate multiple models have been increasingly explored to leverage the strengths of different algorithms. Zhou[15] emphasized that ensemble methods can outperform individual models by reducing variance and bias, thus improving predictive accuracy. A recent study by Fernández-Delgado et al.[16] provided a comprehensive evaluation of different ensemble techniques, confirming their effectiveness in various prediction tasks.

Bagging, or bootstrap aggregating, is a popular ensemble technique that involves training multiple models on different subsets of the data and averaging their predictions. This approach reduces variance and enhances robustness. Barboza et al.[17] highlighted the effectiveness of bagging in improving model stability and accuracy. However, bagging may not fully capture complex interactions in the data, necessitating complementary techniques.

Boosting, another powerful ensemble method, builds models sequentially, with each model focusing on correcting the errors of its predecessors. This iterative improvement can lead to highly accurate predictions. Chen and Guestrin[18] demonstrated the strength of boosting in the form of XGBoost, a scalable and efficient implementation widely used in various applications, including stock price prediction.

Stacking, which involves training multiple base models and using their predictions as inputs to a meta-model, has also shown promise. This technique leverages the strengths of diverse models to improve overall performance. For example, Sagi and Rokach[19] provided a comprehensive survey on ensemble methods, highlighting the potential of stacking to improve predictive performance. However, careful selection of base models and meta-models is crucial to maximize the benefits of stacking.

Recent work by Livieris, Pintelas, and Pintelas[20] explored a CNN-LSTM model for time series forecasting, demonstrating the effectiveness of combining convolutional and recurrent neural networks for financial data. Additionally, Kingma and Ba[21] introduced the Adam optimization algorithm, which has become a standard for training deep learning models due to its efficiency and effectiveness. Another significant contribution is by S. Pawaskar[22], who examined various machine learning algorithms for stock price prediction, highlighting the benefits and limitations of different approaches. Moreover, a study by Nelson, Pereira, and de Oliveira[23] introduced a hybrid model combining LSTM and technical indicators for stock price prediction, demonstrating the effectiveness of integrating different neural network architectures.Finally, Qiu and Song[24] investigated the integration of sentiment analysis and machine learning techniques for stock price prediction, showcasing the added value of textual data in improving predictive performance.

Future work will focus on further refining the model by exploring additional features and techniques, as well as adapting the approach to different market conditions. The integration of alternative data sources, such as sentiment analysis from social media, may provide additional insights and improve predictive performance. By continually advancing the methodology, we aim to contribute to the ongoing development of robust and accurate stock price prediction models.

3. Methodology

Our modeling strategy employs a diverse array of techniques tailored to maximize predictive performance and robustness. Leveraging both traditional and cutting-edge methodologies, our ensemble approach integrates LSTM, GRU, LR, LightGBM. Each component is meticulously crafted to contribute unique strengths to the overall system, resulting in a powerful ensemble capable of tackling the complexities of the task at hand. The whole model ensemble pipeline is shown in Figure 1.

3.1. LSTM

The long-short-term memory (LSTM) network is designed to capture long-term dependencies in sequential data. It addresses the vanishing gradient problem in traditional RNNs by incorporating memory cells that can maintain information over long periods.The LSTM branch processes the input features through several steps:

• Batch Normalization: Standardizes the inputs.
• Dense Layers: Introduces non-linearity and reduces dimensionality.
• Dropout Layers: Prevents overfitting by randomly setting a fraction of input units to 0 at each update during training time.
• Reshape Layers: Prepares the data for LSTM layers.
• LSTM Layers: Processes the sequential data.

The equations governing an LSTM cell are:

$$\begin{array}{cc}\hfill f_t& =\sigma (W_f·[h_{t-1},x_t]+b_f)\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill i_t& =\sigma (W_i·[h_{t-1},x_t]+b_i)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill {\tilde{C}}_t& =tanh(W_C·[h_{t-1},x_t]+b_C)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill C_t& =f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill o_t& =\sigma (W_o·[h_{t-1},x_t]+b_o)\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill h_t& =o_t\ast tanh\left(C_t\right)\hfill \end{array}$$

(6)

where $f_t$, $i_t$, $C_t$, $o_t$, and $h_t$ are the forget gate, input gate, cell state, output gate, and hidden state at time t, respectively.

3.2. GRU

Gated Recurrent Units (GRUs) simplify the LSTM architecture by combining the forget and input gates into a single update gate, making the model faster to train while maintaining performance.The GRU branch processes the input features through:

• Dense Layers: Introduces non-linearity.
• Batch Normalization: Ensures standardized inputs.
• Reshape Layers: Prepares the data for GRU layers.
• GRU Layers: Processes the sequential data.

The equations governing a GRU cell are:

$$\begin{array}{cc}\hfill z_t& =\sigma (W_z·[h_{t-1},x_t]+b_z)\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill r_t& =\sigma (W_r·[h_{t-1},x_t]+b_r)\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\tilde{h}}_t& =tanh(W_h·[r_t\ast h_{t-1},x_t]+b_h)\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill h_t& =(1-z_t)\ast h_{t-1}+z_t\ast {\tilde{h}}_t\hfill \end{array}$$

(10)

where $z_t$, $r_t$, and $h_t$ are the update gate, reset gate, and hidden state at time t, respectively.

3.3. LR

Logistic Regression (LR) is used as a robust baseline model, which is integrated with the deep learning models to enhance stability and reduce overfitting.The logistic regression model is defined as:

$$\begin{array}{c}\hfill \widehat{y}=\sigma (W·X+b)\end{array}$$

(11)

where $\sigma$ is the sigmoid function:

$$\begin{array}{c}\hfill \sigma \left(z\right)={\displaystyle \frac{1}{1+e^{-z}}}\end{array}$$

(12)

This model helps in leveraging linear relationships and serves as a comparative baseline to complex models.

3.4. LightGBM

LightGBM is a gradient boosting framework that uses tree-based learning algorithms. It is designed for high efficiency and scalability.The LightGBM model uses the following key steps:

• Feature Engineering: Creation of new features such as returns, moving averages, and volatility.
• Tree-based Learning: Utilizes decision trees to iteratively improve model performance.

The objective function optimized by LightGBM is:

$$\begin{array}{c}\hfill \mathcal{L}=\sum _{i=1}^{n}l(y_i,{\widehat{y}}_i)+\sum _{j=1}^m\Omega \left(f_j\right)\end{array}$$

(13)

where l is the loss function (e.g., MSE), and $\Omega$ is the regularization term to prevent overfitting. The integration of LightGBM with the deep learning model enhances feature embeddings and overall model performance.

3.5. Loss Function

The model is trained using multiple loss functions to enhance its predictive capabilities:

3.5.1. Mean Squared Error

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2$$

(14)

3.5.2. Correlation Loss

$$\mathrm{Correlation}\mathrm{Loss}=1-{\displaystyle \frac{\mathrm{Cov}(y,\widehat{y})}{{\sigma }_y{\sigma }_{\widehat{y}}}}$$

(15)

3.5.3. Sharpe Ratio Loss

$$\mathrm{Sharpe}\mathrm{Loss}=-{\displaystyle \frac{\mathbb{E}[r-r_f]}{{\sigma }_r}}$$

(16)

where r is the return, $r_f$ is the risk-free rate, and ${\sigma }_r$ is the standard deviation of the return.

3.6. Data Preprocessing

Data preprocessing is a crucial step in ensuring the quality and reliability of the input data. This process involves several steps to handle missing values, outliers, feature scaling, and the computation of technical indicators.Handling missing values and outliers is essential to avoid bias and inaccuracies in the model. The following methods were used

3.6.1. Missing Values

Missing values were replaced using the forward fill method, where the last known value is carried forward to fill the gaps. This method is suitable for time series data as it maintains the continuity of the sequence.

3.6.2. Outliers

Outliers were either removed or capped based on domain-specific rules. For example, stock prices that deviated significantly from the mean were capped to a maximum allowable value to prevent distortion of the model training process.

3.6.3. Feature Scaling and Technical Indicators

Feature scaling and the computation of technical indicators are performed to normalize the data and extract meaningful patterns. The following features were computed: Return (RT):

$$RT={\displaystyle \frac{P_t-P_{t-1}}{P_{t-1}}}$$

(17)

where $P_t$ is the price at time t.

Moving Average (MA):

$$MA={\displaystyle \frac{1}{N}}\sum _{i=0}^{N-1}{P}_{t-i}$$

(18)

where N is the window size.

Exponential Moving Average (EMA):

$$EM{A}_t=\alpha ·P_t+(1-\alpha )·EM{A}_{t-1}$$

(19)

where $\alpha$ is the smoothing factor.

Volatility (V):

$$V=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=0}^{N-1}{(P_{t-i}-MA)}^2}$$

(20)

These technical indicators help in capturing trends and patterns in the stock prices over different time windows.

4. Experimental Results

Evaluation metrics are used to assess the performance of the model. The following metrics were selected:

4.1. Root Mean Squared Error

RMSE is a commonly used metric to measure the differences between predicted and observed values. It is defined as:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}$$

(21)

where $y_i$ is the observed value, ${\widehat{y}}_i$ is the predicted value, and n is the number of observations. RMSE provides a measure of the average magnitude of the errors, giving higher weight to larger errors.

4.2. Correlation Coefficient

The correlation coefficient measures the linear relationship between the predicted and observed values. It is defined as:

$$\mathrm{Correlation}={\displaystyle \frac{\mathrm{Cov}(y,\widehat{y})}{{\sigma }_y{\sigma }_{\widehat{y}}}}$$

(22)

where $\mathrm{Cov}(y,\widehat{y})$ is the covariance between y and $\widehat{y}$, and ${\sigma }_y$ and ${\sigma }_{\widehat{y}}$ are the standard deviations of y and $\widehat{y}$, respectively. A higher correlation coefficient indicates a stronger linear relationship between the predicted and observed values.

4.3. Mean Squared Error

MSE is another metric used to measure the average of the squares of the errors. It is defined as:

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2$$

(23)

MSE is useful for identifying the variance of the prediction errors.

4.4. Performance

The models were evaluated on a public and private test set, with performance measured by the metric we mentioned before. The results are summarizedin Table 1:

The LSTM + GRU + LR + LightGBM model achieved the highest scores.

5. Conclusion

In conclusion, we developed a hybrid deep learning model that combines LSTM and GRU layers with a hyperparameter tuning mechanism. The model effectively processes stock price data and adjusts prices to account for corporate actions. By integrating multiple loss functions and evaluation metrics, the proposed model demonstrates superior performance in predicting stock prices, outperforming traditional models and other deep learning architectures. This work contributes to the advancement of machine learning and deep learning applications in financial time series analysis.