Enhanced Informer-Based Deep Learning for Three-Year ENSO Forecasting

1. Introduction

The El Niño–Southern Oscillation (ENSO) is a major mode of coupled ocean–atmosphere variability and one of the most important sources of interannual climate fluctuations. It is mainly expressed through anomalous sea surface temperature variations in the tropical Pacific Ocean and associated changes in atmospheric circulation. ENSO events, including the warm El Niño phase and the cold La Niña phase, have generally been described as irregular phenomena occurring at intervals of about 2 to 7 years (Wang et al. 2023). Recent observations, however, suggest that ENSO variability may be changing under the influence of a warming climate. In particular, Central Pacific El Niño events have occurred more frequently since 2000, and their dominant recurrence period has shortened from approximately 4–5 years to 2–3 years (Jia and Guo 2025). This change points to a less stationary ENSO regime and highlights the need for forecasting methods that can remain reliable under evolving climate conditions.

The practical importance of ENSO prediction has also become more evident in recent years. The strong 2023–2024 El Niño event, together with record greenhouse gas concentrations and other contributing factors, contributed to the exceptional warmth of 2024, which was confirmed as the warmest year on record (WMO 2025). In addition, the World Meteorological Organization reported an 80% likelihood of El Niño conditions during June–August 2026, with probabilities near or above 90% for continuation until at least November 2026 (WMO 2026a). Recent European heat extremes further underline the practical value of improved climate early-warning systems. Europe experienced severe heatwave conditions in 2025, and the late-June 2026 heatwave broke numerous temperature records while affecting human health, ecosystems, agriculture, infrastructure, and labour productivity (WMO 2026b; WMO 2026c). According to the World Meteorological Organization, more than 1,300 excess deaths had been recorded across Europe since 21 June in association with the extreme heat, while preliminary national mortality estimates later indicated at least 3,700 excess deaths during the June heatwave in France, Belgium, and the Netherlands alone (WMO 2026d; Reuters 2026). Although European heatwaves cannot be attributed to ENSO alone, ENSO-related teleconnections can influence atmospheric circulation and climate anomalies outside the tropical Pacific. Therefore, improving long-term ENSO forecasting is essential for climate-risk preparedness and for decision-making in agriculture, water-resource management, disaster-risk reduction, public health, and energy planning (Scaife 2010; Blöschl et al. 2019; L’Heureux et al. 2020).

The use of machine learning has introduced new methods to address these long-standing problems.

However, current dynamic forecasting models continue to be incapable of providing reliable forecasts greater than a single year; thus, predicting ENSO events for multiple years remains a challenge.

Despite the difficulties that still exist; it has been possible to identify periodic variations and regular cycles associated with the ENSO phenomenon as well as gradual changes in oceanic variability which indicate that the ENSO phenomenon is potentially predictable (Luo et al. 2008).

Anomalies in the equatorial Pacific Ocean associated with several of the recent La Niña events have shown extended time periods of relatively low activity, indicating a change in the ENSO paradigm (Gao and Zhang 2017) that makes it more difficult to make accurate forecasts today compared with the forecast skill seen in the 1980s and 1990s.

By leveraging the potential of machine learning and integrating recent discoveries regarding the teleconnections of the El Niño–Southern Oscillation (ENSO) with other oceanic systems, it is feasible to develop a model capable of forecasting ENSO events beyond a one-year timeframe (Park et al. 2017). Sea Surface Temperature (SST) anomalies occurring outside the equatorial Pacific can indeed signal the onset of ENSO events more than a year in advance. Concurrently, advanced large-scale numerical models have been developed and implemented to elucidate the determinants of Earth's historical climate, thereby enhancing our capacity to predict future climatic conditions (Kirtman and Power 2013). While there is a relatively strong consensus among climate models concerning large-scale temperature signals, uncertainties surrounding dynamical alterations in atmospheric circulation remain significant. This discrepancy inevitably undermines the reliability of long-term forecasts, particularly concerning precipitation projections for the coming decades (Shepherd 2014).

The Multimodal ENSO Forecast (MEF), introduced in our previous work and subsequently evaluated in related studies (Naisipour et al. 2024, 2025a, 2026), combines spatiotemporal oceanic fields with a time-series Informer module to predict ENSO up to 24 months ahead.

Subsequent analysis using ablation revealed that the model's long-range predictive skill (particularly for periods greater than 18 months) was driven almost entirely by the time-series branch through an Informer-based architecture. By comparison, the contribution of high-dimensional spatial input was minimal while imposing considerable computational expense on the model. This finding illustrates that when predicting ENSO beyond 18-month lead times, temporal dynamics provide stronger predictive signals than high-dimensional spatial anomaly patterns, supporting a transition away from multi-modal architectures to time-series-only architectures.

Based on these findings, we propose a more efficient forecasting framework that eliminates unnecessary spatial processing and focuses on one-dimensional climate time-series signals. The Informer architecture (Zhou et al., 2021) is great for doing this since it was created to predict time series of long sequences using an attention mechanism called ProbSparse Self-Attention. This attention method gives more emphasis on the time steps which are important by using their locations and attention weights to make its decisions. The Informer can capture short-term auto-correlations as well as multi-year periodicity in the ENSO time-series without needing to do a lot of feature engineering and without having to find prior clues from outside the model. This extensive dataset allows the deep learning model to learn from a wide range of scenarios, improving its predictive performance.

To enhance the model’s ability to learn ENSO dynamics under different climate backgrounds, we expanded the training dataset by combining historical simulations from CMIP5 and selected CMIP6 models with observational records. Unlike a simple inclusion of all available simulations, the CMIP6 models were screened based on their ability to reproduce key ENSO characteristics, including Niño 3.4 variability, dominant periodicity, seasonal phase locking, and the observed tendency toward more frequent Central Pacific El Niño events after 2000. This selection strategy is consistent with recent model-evaluation studies showing that ENSO-related metrics can be used to diagnose the suitability of coupled climate models for ENSO analysis and prediction (Planton et al. 2021; Jia and Guo 2025). By incorporating CMIP6 simulations that better represent ENSO frequency and variability, the training set covers a broader range of physically plausible ENSO regimes and improves the model’s generalization to the post-2000 climate period.

The model was then retrained and calibrated using Global Ocean Data Assimilation System (GODAS) observations from 1980 to 2000.

The proposed streamlined framework provides several advantages compared with traditional 3D convolutional neural networks (3DCNNs) and other spatially intensive deep-learning approaches. By removing the 3DCNN component from the architecture, the model substantially reduces computational demand and can perform inference without GPU acceleration. This simplified design also eliminates the need to preprocess and store large spatial anomaly maps, reducing both storage requirements and data-management costs. In addition, the modified architecture contains fewer hyperparameters, which makes systematic optimization easier and reduces the risk of overfitting when working with climate datasets that have limited independent sample sizes.

Similarly, in time series forecasting, multimodal techniques have been employed to analyze heterogeneous data streams, such as meteorological and hydrological indicators, to enhance predictive performance. Kumshe et al. (2024) leveraged CNN-LSTM pipeline to capture spatio-temporal data for forecasting hydrological phenomena. In climate science, recent studies have prioritized the prediction of the ENSO through the utilization of spatio-temporal data. Zhao et al. (2023) applied a deep learning model that integrates satellite imagery and oceanographic temporal data, capturing the intricate interactions that influence ENSO events and resulting in notable improvements in prediction skill. Jonnalagadda and Hashemi (2023) explored the use of recurrent neural networks to enhance the prediction of El Niño and La Niña events by optimizing the spatial-temporal extent of environmental factors, demonstrating the potential of advanced modeling techniques in improving climate predictions.

For quantitative benchmarking, we compare our proposed model against two established references: the SINTEX-F dynamical model (Doi et al. 2019) and the CNN-based model of Ham et al. (2019). The benchmark to quantitatively compare our model with regards to performance of forecasting the ENSO is based on the Convolutional Neural Network developed by Ham et al. (2019), which is recognized widely as being one of the best currently available deep-learning predictive models of ENSO. The Ham et al. model uses as input a map of sea surface temperature (SST) and a map showing deviations in ocean heat content (HC), and applies three (2D) convolutional layers in order to derive spatially based features in order to predict the Niño 3.4 index. Although the Ham et al. predictive model has exhibited skill that exceeds the SINTEX-F (Doi et al., 2019) model during the periods covered by these models, (i.e., lead times less than 18 months), the performance of the model (prediction capabilities) greatly diminishes at lead times beyond 18 months when the model's correlation coefficient would be less than 0.50. Most importantly, Ham et al. (2019) model produced poor predictive accuracy for the time period after year 2000. The relationship between the increased frequency of ENSO and the diminished number of spatially reliable "precursory pattern" for the ENSO means that there will be less reliable predictions of the ENSO based on the use of static spatial pattern features. Therein lies a strong argument to explore alternative architectural designs that require less spatially static (i.e., based on using a reliable static feature space) and instead rely extensively on the use of more reliable temporal dynamic relationship to predict the ENSO.

Ham et al. addressed the spring predictability barrier by introducing the ACNN method, a variant of the conventional Convolutional Neural Network (CNN) approach, which incorporates modifications to the architectural design; nevertheless, the overall correlation skill remained consistent (Ham et al. 2021). Zhou et al. posited that by employing the Principal Oscillation Pattern analysis in conjunction with CNN-Long Short-Term Memory (LSTM) (Muhammad et al. 2023) techniques, they could predict the Niño 3.4 index for up to 17 months during the validation period spanning from 1994 to 2017 (Zhou and Zhang 2022). Moreover, Zhou and Zhang (2023) proposed an innovative application of the Transformer architecture, termed 3D-Geoformer, designed for spatiotemporal multivariate predictions of ENSO. This model integrates sea surface zonal and meridional wind stress alongside seven-layer ocean temperature anomalies within the upper 150 meters, employing a three-dimensional framework. The 3D-Geoformer effectively hindcasts the Niño 3.4 index with a lead time of up to 18 months, utilizing data from 1983 to 2021. However, it is important to note that the model yields valid results only for predictions made up to 16 months in advance during the boreal spring. In comparative analyses, the performance of the 3D-Geoformer model was evaluated against that of the CNN model for the period from 1983 to 2017, revealing only marginal improvements in overall forecast skill. Similar to the CNN model, the skill of the 3D-Geoformer post-2000 is relatively low, with its correlation coefficient declining below 0.5 for lead times extending to 13 months.

Recent studies published in high-impact journals have further emphasized the rapid development of hybrid, Transformer-based, and physically informed deep-learning approaches for ENSO prediction. For example, Mu et al. (2024) introduced ENSO-PhyNet, a Transformer-based model that incorporates heat-budget dynamics into self-attention computations to improve physical interpretability. Chen et al. (2025) proposed a combined dynamical–deep learning framework, showing that deep learning can complement dynamical forecast systems and improve ENSO prediction skill. More recently, Zhou et al. (2026) demonstrated that explicitly accounting for tropical basin interactions can reduce the spring predictability barrier and extend effective ENSO prediction beyond 16 months. These studies confirm the growing importance of physically informed and hybrid deep-learning frameworks, while also showing that reliable prediction beyond about 18–24 months remains a major challenge.

Collectively, these advancements highlight the potential of multimodal deep learning approaches to significantly enhance the understanding and prediction of ENSO phenomena, thereby contributing to more effective climate monitoring and response strategies. Despite these advancements, achieving a correlation skill greater than 50% in predicting ENSO events beyond a 12-month horizon continues to pose significant challenges. Given the increasing variability of ENSO, no existing methodology has demonstrated the capacity to predict ENSO occurrences with reliable accuracy at a lead time of 1.5 years.

The increasing impact of climate change upon the dynamics of ENSO leads to more frequent occurrences of extreme El Niño and La Niña phenomenon (Cai et al. 2014, 2020). The evolution of ENSOs’ behaviour under non-stationary climate conditions presents complications for prediction capabilities using traditional modelling techniques, including both dynamical and statistical modelling processes. The inadequate accuracy of predictions has considerable societal repercussions, such as negative effects on agricultural output, management of water resources, and preparation for disaster (L'Heureux et al. 2020). Due to these factors, there is an urgent need to develop new forecasting frameworks capable of predicting changing climate systems; currently existing static spatial precursor patterns may not be applicable to future post-2000 climatic conditions.

The article by Ricke and Caldeira (2014) examines the interplay between natural climate variability, particularly the ENSO, and human-induced climate change, highlighting ENSO's significant role in influencing global weather patterns and climate extremes. The authors underscore the critical importance of accurately distinguishing between the effects of natural variability and those of anthropogenic climate change for effective climate predictions and policies. Consequently, they advocate for further research to enhance our understanding of these dynamics, which is paramount for refining climate models and predicting the ENSO phenomena in the new millennia. In this study, the post-2000 period was chosen exclusively for validation due to the observed increase in the frequency of central Pacific El Niño events and the associated weakening of warm water volume variations, which have complicated ENSO predictions (Guan and McPhaden 2016; Wang et al. 2024). To address this challenge, we developed the enhanced Informer-based forecasting framework described in this paper.

The improved framework presents four substantial contributions. Our first contribution demonstrates the effectiveness of a time-series-only deep learning model that outperforms the existing state of the art with lead times of 36 months (the current state of the art is unable to perform well beyond 18 months). Our second contribution has shown that combining historical data from all CMIP5 and CMIP6 models leads to significantly improved long-lead forecasting skill than if training on only CMIP5 or limited historical observations, therefore providing a way to augment climate forecasting data with historical data. Our third contribution provides a new understanding from an ablation analysis that the high-dimensional spatial input to the model has virtually no impact in forecasting skill beyond 18 months, contrary to previous beliefs that long-term forecast skill required the inclusion of high spatial complexity. Our fourth contribution establishes an entirely new efficient benchmark for operational long-lead ENSO forecasting and shows that an optimized Informer-type forecasting model using only one-dimensional climate data is superior to far more complex and computationally intensive forecasting techniques, such as numerical dynamic models and deep learning techniques that use image data. Together, these contributions redefine the state of the art in long-lead ENSO prediction and offer a replicable framework for extending time-series-based forecasting to other climate phenomena exhibiting multi-year variability.

2. Method

The single streamlined module of the enhanced Time Series Informer (TSI) network is the proposed ENSO forecasting framework. In contrast to our Multimodal ENSO Forecast (MEF) framework, which included both an Informer for time series data and a 3D Convolutional Neural Network (3DCNN) for processing the spatial component of ENSO, this model will only process one-dimensional climate input. This decision was made based on our ablation study results, which showed that using high-dimensional input data does not increase the accuracy of forecasts beyond 18 months and comes at a high computational cost. In the next sections, we will provide more information about: 1) how the TSI module works; 2) our data augmentation strategy using CMIP5 and CMIP6; 3) how we prepared the input data; and 4) how we will evaluate the model.

2.1. Time Series Informer

A possible means of addressing the ongoing development of El Nino Southern Oscillation (ENSO) in a deep learning framework is to consider an extended sequence of anomalies as an input to the deep learning models. To be effective in practice, the number of parameters becomes very large, making prediction of longer than short-term periods impossible. In addition, large numbers of parameters will make it very difficult for the model to use extraneous elements in very long input sequences. Conversely, short input sequence lengths may not capture sufficient dimension of time series data to adequately represent the full nature of the system. One viable approach for generating long-term temporal characteristics is to use the independent examination of ENSO's temporal characteristics. We will use the El Nino time series (ENSO) as input to the Time Series Informer module (TSI) to predict its long-term temporal characteristics.

Transformer neural networks, as introduced by Vaswani et al. (2017), are advanced machine learning models designed primarily to mitigate the limitations associated with LSTM networks. The foundational principle of this approach lies in its ability to attend not only to events occurring prior to a specific predictand but also to those that transpire subsequently. This dual focus has garnered considerable interest among researchers across diverse fields, including information retrieval, text classification, and document summarization. Beyond applications related to language, Transformers have also been effectively employed in disciplines such as computer vision (Nicolas et al. 2020), chemistry (Schwaller et al. 2019), life sciences (Rives et al. 2016), and river level forecasting (Castangia et al. 2023).

One of the more recent implementations of Transformers is in the context of ENSO forecasting, as demonstrated by Ye et al. (2022). They assert that their model can predict ENSO events with greater accuracy than convolutional neural networks (CNNs) up to one and a half years in advance. Nevertheless, the model exhibits certain limitations, notably a relatively low predictive skill for strong ENSO events. Furthermore, predictions are notably more accurate in the spring season compared to those generated by other leading models.

Inspired by the work of Zhou et al. (2021), we feed long-term ENSO time series into the Informer network and receive predictions extending up to 36 months.The overall architecture of Informer network ${\mathcal{G}}_{{\theta }_G}$, as shown in Fig. 1, consists of five main components.

Figure 1. Overall view of Informer Module.

Figure 1. Overall view of Informer Module.

Given a time sequence ${\mathcal{X}}^t=\left\{{\mathrm{x}}_1^t,\text{…},{\mathrm{x}}_{L_x}^t|{\mathrm{x}}_i^t\in {\mathbb{R}}^{dx}\right\}$ at time t and the output corresponding sequence ${\mathcal{Y}}^t=\left\{{\mathrm{y}}_1^t,\text{…},{\mathrm{y}}_{L_x}^t|{\mathrm{y}}_i^t\in {\mathbb{R}}^{dy}\right\}$, the module encodes the input representations ${\mathcal{X}}^t$ into a hidden state representation ${\mathcal{H}}^t$ and decodes an output representations ${\mathcal{Y}}^t$ from ${\mathcal{H}}^t=\left\{{\mathrm{h}}_1^t,\text{…},{\mathrm{h}}_{L_h}^t\right\}$. The extracted features of each series are added to the positional encoders, after the input series have been embedded and representative features have been created by the embedding module. We consider that we have t-th sequence input ${\mathcal{X}}^t$ and p types of global time stamps and the feature dimension after input representation is $d_{model}$. We first use a fixed position embedding to maintain the local context:

$${\mathbf{P}\mathbf{E}}_{\left(\mathit{p}\mathit{o}\mathit{s},\mathsf{2}\mathit{j}\right)}=\sin \left(\mathit{p}\mathit{o}\mathit{s}/{\left(\mathsf{2}{\mathit{L}}_{\mathit{x}}\right)}^{\mathsf{2}\mathit{j}/{\mathit{d}}_{\mathit{m}\mathit{o}\mathit{d}\mathit{e}\mathit{l}}}\right)$$

(2)

$${\mathbf{P}\mathbf{E}}_{\left(\mathit{p}\mathit{o}\mathit{s},\mathsf{2}\mathit{j}+\mathsf{1}\right)}=\cos \left(\mathit{p}\mathit{o}\mathit{s}/{\left(\mathsf{2}{\mathit{L}}_{\mathit{x}}\right)}^{\mathsf{2}\mathit{j}/{\mathit{d}}_{\mathit{m}\mathit{o}\mathit{d}\mathit{e}\mathit{l}}}\right)$$

(3)

Where $\mathcal{j}\in \left\{\mathsf{1},\dots ,{\mathit{d}}_{\mathit{m}\mathit{o}\mathit{d}\mathit{e}\mathit{l}}/\mathsf{2}\right\}$. A learnable stamp embedding ${\mathbf{S}\mathbf{E}}_{\left(\mathit{p}\mathit{o}\mathit{s}\right)}$ utilizes every global time stamp with limited vocab size. Since the self-attention’s similarity computation can have access to global context, the computation consuming is affordable on long inputs. we project the scalar context ${\mathbf{x}}_{\mathit{i}}^{\mathit{t}}$ into ${\mathit{d}}_{\mathit{m}\mathit{o}\mathit{d}\mathit{e}\mathit{l}}$-dim vector ${\mathbf{u}}_{\mathit{i}}^{\mathit{t}}$ with 1-D convolutional filters with kernel width equal to 3 and stride one to align the dimension. Thus, we have the feeding vector:

$${\mathcal{X}}_{\mathit{f}\mathit{e}\mathit{e}\mathit{d}\left[\mathit{i}\right]}^{\mathit{t}}=\mathit{\alpha }{\mathbf{u}}_{\mathit{i}}^{\mathit{t}}+{\mathbf{P}\mathbf{E}}_{\left({\mathit{L}}_{\mathit{x}}\times \left(\mathit{t}-\mathsf{1}\right)+\mathit{i},\right)}+\sum _{\mathit{p}}{\left[{\mathbf{S}\mathbf{E}}_{\left({\mathit{L}}_{\mathit{x}}\times \left(\mathit{t}-\mathsf{1}\right)+\mathit{i}\right)}\right]}_{\mathit{p}}$$

(4)

In this equation, $\mathit{\alpha }$ is the factor balancing the magnitude between the scalar projection and local/global embedding that is set to 1 in this study, and $\mathcal{i}\in \left\{\mathsf{1},\dots ,{\mathit{L}}_{\mathit{x}}\right\}$.

Figure 2. The encoder’s single stack in the Informer module.

Figure 2. The encoder’s single stack in the Informer module.

ProbSparse Self-attention. We apply the ProbSparse self-attention by allowing each key to only attend to the u dominant queries:

$$\mathcal{A}\left(\mathrm{Q},\mathrm{K},\mathrm{V}\right)=\mathrm{Softmax}\left({\displaystyle \frac{\stackrel{\mathrm{-}}{\mathrm{Q}}{\mathrm{K}}^{\mathrm{T}}}{\surd d}}\right)V$$

(5)

Where Q is a sparse matrix of the same size of q and it only contains the Top-u queries under the sparsity measurement $M\left(\mathrm{q},\mathrm{K}\right)$.

Encoder. The encoder's purpose is to extract from the lengthy sequential inputs the reliable long-range dependency. As the input representation is done, the matrix ${\mathrm{X}}_{en}^t\in {\mathbb{R}}^{L_x\times d_{model}}$ will be the new shape of the t-th sequence input ${\mathcal{X}}^{\mathit{t}}$.

Self-attention Distilling. We use the distilling operation to privilege the superior ones with dominating features, because the natural consequence of the ProbSparse self-attention mechanism has redundant combinations of value $\mathrm{V}$. This also makes a focused self-attention feature map in the next layer. Observing the n-heads weights matrix of the Attention blocks in Fig. 2, it drastically reduces the input's time dimension. According to equation (6), the applied procedure forwards from j-th layer into (j + 1)-th layer.

$${\mathrm{X}}_{j+1}^t=\mathrm{MaxPool}\left(\mathrm{ELU}\left(\mathrm{Conv}1\mathrm{d}\left({\left[{\mathrm{X}}_{\mathrm{j}}^{\mathrm{t}}\right]}_{\mathrm{AB}}\right)\right)\right)$$

(6)

In this formula, ${\left[·\right]}_{AB}$ represents the attention block, $\mathrm{ELU}\left(·\right)$ is the activation function and $\mathrm{Conv}1\mathrm{d}\left(·\right)$ is an 1D convolutional filter with kernel width of three. To reduces the whole memory usage, a max-pooling layer with stride 2 down samples $X^t$ into its half slice.

Decoder. We use a stack of two identical multi-head attention layers as the decoder, and to alleviate the speed plunge in long prediction, we apply the generative inference. The following vectors are given to the vanilla decoder as the input:

$${\mathrm{X}}_{\mathrm{de}}^t=\mathrm{Concat}\left({\mathrm{X}}_{\mathrm{token}}^t,{\mathrm{X}}_0^t\right)\in {\mathbb{R}}^{\left(L_{\mathrm{token}}+L_y\right)\times d_{model}}$$

(7)

where ${\mathrm{X}}_{\mathrm{token}}^t\in {\mathbb{R}}^{L_{\mathrm{token}}\times d_{\mathrm{model}}}$and ${\mathrm{X}}_0^t\in {\mathbb{R}}^{L_y\times d_{\mathrm{model}}}$ are is the start token and placeholder for the target sequence, respectively. The final layer is a dense layer that generates the results.

Generative Inference. Start token is efficiently applied in NLP’s “dynamic decoding” (Devlin et al. 2018), and we extend it into a generative way. We sample a $L_{\mathrm{token}}$ long sequence in the input sequence, so ${\mathrm{X}}_0$ contains target sequence’s time stamp, that is the context at the target. Instead of the cumbersome “dynamic decoding” in the vanilla encoder-decoder blocks, the decoder used in this study forecasts outputs by a forward procedure. We chose the MSE loss function across the entire Informer module.

Evaluation metrics. To evaluate the predictive capability of the proposed model for forecasting the ENSO, we computed the temporal anomaly correlation coefficient (ACC), which is defined as follows:

$${\mathrm{A}\mathrm{C}\mathrm{C}}_{l,m}={\displaystyle \frac{{\displaystyle {\sum }_{y=s}^e}{\widehat{E}}_{y,m,l}^{\text{'}}{E}_{y,m}^{\text{'}}}{\sqrt{{\displaystyle {\sum }_{y=s}^e}{{\widehat{E}}_{y,m,l}^{\text{'}}}^2{\displaystyle {\sum }_{y=s}^e}{E_{y,m}^{\text{'}}}^2}}}$$

(8)

Here, $\widehat{E}$ is the predicted version of $E$. $E^{\text{'}}$ denotes the difference between $E$ and the climatology, that is, the long-term mean of weather states that are estimated on the training data. Here, m and l are the calendar month (from 1 to 12) and the forecast lead months (1-36), respectively. The label y denotes the forecast target year. Finally, s and e denote the earliest (that is, 2001) and the latest year (that is, 2020) of the validation, respectively.

For deeper insight, we evaluated the method using Root Mean Square Error (RMSE) as follows:

$${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{l,m}={\displaystyle \frac{{\displaystyle {\sum }_{y=s}^e}{\left({\widehat{E}}_{y,m,l}^{\text{'}}-E_{y,m}^{\text{'}}\right)}^2}{\left|e-s\right|}}$$

(9)

Hyperparameters. The hyperparameters used in this study for the TSI module are as follows: number of encoder layers = 3, number of decoder layers = 2, embedding dimension $d_{model}$ = 512, number of attention heads = 8, dropout rate = 0.1, batch size = 32, and learning rate = 0.0001 with the Adam optimizer. These values were determined through systematic optimization using the ROA method (Talaat and Gamel 2022).

2.2. Implementation of Code and Cloud Super Computing

Python and TensorFlow (Abadi et al. 2016) were used to implement the proposed model. Part of the coding, debugging, and experimental execution was carried out in a cloud-based computational environment using Google Colaboratory, which provides a serverless Jupyter notebook interface with access to GPU/TPU acceleration for machine-learning workflows (Bisong 2019). The use of a streamlined time-series-only architecture substantially reduced the computational requirements compared with the earlier MEF framework. During model development, cloud resources were used for training and hyperparameter testing, while the final trained model could perform inference on a standard CPU within seconds. This makes the proposed framework suitable for reproducible research and future operational deployment without requiring a dedicated high-performance computing cluster.

2.3. Data Augmentation with CMIP5 and CMIP6

An innovative aspect of the new framework includes the systematic incorporation of historical simulations from the coupled model intercomparison project (CMIP5 & CMIP6). Previous studies examining ENSO using deep learning have either depended on a combination of CMIP5-only, as in Ham et al. (2019), or limited observational data when incorporating multiple model generations into prediction algorithms. The inclusion of CMIP5/CMIP6 brings three primary benefits:

First, the newer CMIP6 contains the most up-to-date historical simulations with improved physical parameterizations and an increase in spatial resolution compared with CMIP5. Secondly, the combined ensembles of CMIP5/CMIP6 will include a larger spread of variability for the ENSO phenomenon, resulting in a larger representation of rare extreme events than possible with only CMIP5. Third, training on multiple model generations will decrease the risk of overfitting the data, thus improving generalization of the model to observational data.

Compared to the earlier MEF framework, the current framework does not use the 3DCNN module; the entire training of the model is conducted with the TSI module only, using one-dimensional time series as input (primarily the monthly Niño 3.4 index, with other potential auxiliary indices such as IOD or PMM possibly augmenting the time series). The CMIP5 and CMIP6 data repositories are displayed in Table 1.

2.4. Input and Output Data

The input to the TSI module is the Niño 3.4 Index. This is the average sea surface temperature anomaly for the area between 120°W and 170°W and 5°S and 5°N. The Niño 3.4 Index is the way to describe El Niño and La Niña events. The TSI module uses this index to predict the Niño 3.4 Index for the 1 to 36 months.

The TSI module was trained using simulations from CMIP5 and CMIP6. These simulations are from 1850 to 2000. There are 21 CMIP5 models and 30 CMIP6 models. This gives us a lot of data to work with. The TSI module was fine-tuned using data from GODAS from 1980 to 2000. Then it was tested on GODAS data from 2001 to 2020. This way we know the TSI module is working well.

Our new model is different from the MEF framework. It only uses time-series data. It does not use maps of sea surface temperature anomalies. This makes it easier to store and process the data. We got 500 GB of raw data from CMIP5 and CMIP6. We used this data to get the Niño 3.4 Index for each model. We checked the data against data from GODAS and against processed data from Ham et al.

Table 1 shows where we got the data, from and where it is stored.

3. Results and Discussion

3.1. Overall Forecasting Skill

The proposed Informer-based prediction model was tested using GODAS observational data (2001-2020) and trained using merged CMIP5/CMIP6 historical data (1850-2000) and fine-tuned to GODAS (1980-2000) maintaining absolute separation between the two in terms of training and validating data.

The current Informer-based prediction model compared to previous MEF model uses only 1D time series data without complex spatial processing or ensemble voting. This architecture produces unprecedented long range skill.

The proposed Informer-based model demonstrates strong predictive skill at short and intermediate lead times, with ACC values decreasing gradually during the first 14 months. Beyond this period, the prediction skill remains relatively stable, fluctuating around 0.4–0.5 up to the 36-month lead time. Although the correlation decreases with increasing lead time, the model preserves useful predictive information over multi-year horizons. These results highlight the potential of deep learning approaches for extending ENSO prediction toward multi-year forecasting horizons.

On the other hand, The CNN benchmark shows a faster degradation of predictive skill compared with the proposed model. According to the available CNN results, the correlation decreases considerably after approximately 18 months and remains lower than the proposed framework at longer lead times. Similarly, the SINTEX-F dynamic model (Doi et al. 2019) degraded to less than 0.4 one month beyond the 18 month prediction period.

3.2. Seasonal Dependence and the Spring Predictability Barrier

The spring predictability barrier remains evident in the proposed model, although its effect is reduced compared with conventional dynamical and deep-learning approaches. At a 24-month lead time, the proposed model maintains ACC values above 0.45 for March–April–May targets. A direct comparison with the CNN benchmark is not reported at this lead time because corresponding CNN values were unavailable.

The decrease in the spring predictability barrier produced by the Informer is due to its use of a ProbSparse self-attention mechanism that allows it to choose the most informative time steps from the entire input sequence and, therefore, avoid the degradation of the signal relative to noise that has typically impacted the prediction of the ENSO during the boreal springtime.

3.3. Comparison with Benchmark Models

To evaluate the proposed model against an established deep-learning benchmark, its ACC values were compared with those of the CNN model developed by Ham et al. (2019) at selected forecast lead times (Table 2). The two models show similar skill at short lead times, but their performance diverges as the forecast horizon increases. The proposed model retains substantially higher ACC values at 12 and 18 months, indicating greater robustness at extended lead times. CNN results beyond 18 months were not available and were therefore neither plotted nor extrapolated.

The improved long-lead performance is consistent with the temporal focus of the Informer architecture. ProbSparse self-attention enables the model to identify informative dependencies across long input sequences, whereas the CNN benchmark relies primarily on spatial features. This distinction may explain the larger performance gap observed at longer lead times.

Figure 3 presents the values reported in Table 2 graphically. At lead times of 1 and 6 months, the differences between the two models are small, at 0.01 and 0.03, respectively. The gap increases to 0.08 at 12 months and 0.29 at 18 months. At the 18-month lead time, the proposed model achieves an ACC of 0.46, compared with 0.17 for the CNN benchmark. At 24 and 30 months, only the proposed model is shown because corresponding CNN results were unavailable.

3.4. Ablation Analysis: The Role of CMIP6 Data

An ablation experiment was performed where the Informer architecture was applied to train on only the CMIP5 data for performance comparison against the CMIP6 results. Other hyperparameters, training methods, and validation periods remained the same.

The model utilizing CMIP5 alone obtains ACC measurements of over 0.5 only for a period of 28 months. After that period, the ACC measurement declines further to a reading of 0.45 at 36 months. However, ACC measurement obtained from the full model of CMIP5 + CMIP6 maintains higher ACC values across long lead times, with the largest improvements observed during the 24–30 month prediction range.

When CMIP6 is added to the model, all lead times demonstrate higher performance than would have been shown with only CMIP5. Specifically, there are about 0.09 more ACC points of gain for lead time periods occurring between 24 and 30 months than what would have been shown based on CMIP5 only. This is most impressive given the fact that the lead time periods between 24 and 30 months are caused by factors that have been associated with major declines in skill from more traditional methodologies (which include both dynamical systems and previously implemented deep-learning based procedures).

These findings confirm three major points. First, adding CMIP6 to the dataset provides a strong advantage in terms of long-lead forecast skill through the combined training of the two datasets than only training with CMIP5. Second, the advantage of using CMIP6 data becomes more pronounced as lead time increases, supporting the hypothesis that the additional variability regimes captured by the CMIP6 dataset provide strong information for anticipating multi-year dynamics associated with the phenomena of ENSO. Third, improved performance during the period of validation that followed 2000 shows that having training from a wider ensemble of different climatic forecast model types provides better generalization ability for the new observational states of ENSO due to increased variability in ENSO over recent times.

3.5. Computational Efficiency

The proposed Informer model requires approximately 8 GPU hours for training, compared with 72 GPU hours for the original MEF framework and 12 GPU hours for the CNN benchmark. Its inference time is approximately 0.5 s per forecast, and inference can be performed on a standard CPU. By contrast, the original MEF framework and the CNN benchmark require GPU acceleration and have inference times of approximately 15 s and 2 s per forecast, respectively. The proposed model also requires only about 2 GB of GPU memory, compared with 45 GB for MEF and 8 GB for the CNN benchmark.

Table 3. Computational efficiency comparison.

Metric	Proposed Model	MEF (original)	CNN (Ham et al. 2019)
Training time (GPU hours)	8	72	12
Inference time (seconds per forecast)	0.5	15	2
GPU memory (GB)	2	45	8
CPU inference possible?	Yes	No	No

Table 3. Computational efficiency comparison.

Metric	Proposed Model	MEF (original)	CNN (Ham et al. 2019)
Training time (GPU hours)	8	72	12
Inference time (seconds per forecast)	0.5	15	2
GPU memory (GB)	2	45	8
CPU inference possible?	Yes	No	No

4. Conclusion

We utilized the Multimodal ENSO Forecasting (MEF) approach to analyze previous findings and determine the driving factors of the forecast's long-range skill (beyond 18 months), using an ablation analysis method to conclude that the Informer time-series branch provided the majority of the long-range skill, with little contribution from the higher-dimensional spatial processing branch. With this new understanding, we re-evaluated the need for spatial complexity in the long-term forecast of ENSO events.

Based on that conclusion, we developed a new, simplified and improved Informer-only based predictive system that relies solely on inputting unidimensional climate signals. The combination of enhanced input representations, systematic hyperparameter optimization and expansion of the training set to include historical simulations (CMIP5 and CMIP6) has enabled the new model to produce unprecedented predictive ability.

The key findings of this study are as follows:

First, the proposed model maintained useful predictive skill at lead times of up to 36 months. Although ACC declined as the forecast horizon increased, it remained approximately within the 0.4–0.5 range at extended lead times and exceeded the benchmark models where directly comparable results were available. To our knowledge, this represents one of the longest lead times reported for deep-learning-based ENSO forecasting.

The results support this conclusion in that a larger dataset trained on different generations of climate models helps the model generalise better to the time period after 2000 when the variability of ENSO behaviour has increased.

Secondly, using only the Informer architecture requires very little compute; it takes about 8 hours of training time on one GPU, and inference can be completed in a few seconds on a regular CPU. In comparison, our original MEF framework used 72 GPU hours and needed 45 GB of memory, and other image-based deep learning models require more resources than this.

Thirdly, the lower spring predictability barrier in our model, which maintains correlations greater than 0.45 for MAM targets even at 24 month lead times, demonstrates that the ProbSparse attention mechanism is effective at capturing seasonal signatures that conventional models cannot retain.

In the future, the way forward will involve two main areas of focus: The integration of additional climate indices (e.g., IOD, PMM) into the model as auxiliary input signals, and extending the operational forecasting option based on transformer architecture beyond 36 months lead.

In general, the Informer Framework has established a new baseline for forecasting operational-long-lead ENSO's (El Niño-Southern Oscillation) using the best blend of accuracy, efficiency, and demonstrated skill to create a scalable and repeatable approach to time-series based climate prediction for those up to 36 months lead. Together this will demonstrate the potential of time-series learning architectures designed with climate-model informed training will significantly advance the development of long-lead climate predictions.