KOSLM: A Kalman-Optimal Hybrid State-Space Memory Network for Long-Term Time Series Forecasting

1. Introduction

Long-term time series forecasting (LTSF) aims to predict future values over extended horizons based on historical observations. Accurate long-range forecasts are critical for applications such as energy scheduling, climate modeling, financial planning, traffic management, and healthcare resource allocation. LTSF presents unique challenges: capturing long-range dependencies, mitigating error accumulation, and adapting to non-stationary temporal dynamics [1].

Traditional recurrent architectures, including Recurrent Neural Networks (RNNs) [2] and Long Short-Term Memory (LSTM) networks [3,4], often struggle with long sequences due to vanishing or exploding gradients [5]. While LSTM gates alleviate some of these issues, their heuristic design lacks explicit structural constraints derived from a principled optimality criterion, potentially leading to suboptimal memory retention over extended horizons.

The Kalman filter (KF) [6] provides optimal state estimation under Gaussian noise [7]. Recent studies have explored integrating KF with LSTM to improve the accuracy of time-series forecasting. Representative approaches include: (i) Deep Kalman Filters [8], which parameterize the state-transition and observation functions of KF using LSTM; (ii) KalmanNet [9], which employs an LSTM to learn residual corrections to the Kalman gain under a known KF model; and (iii) uncertainty-aware LSTM–KF hybrids [10], which estimate the covariance or uncertainty structures of KF through recurrent dynamics. However, these approaches generally maintain a loose coupling between LSTM and KF — they do not embed Kalman-inspired feedback directly into the internal gating dynamics of LSTM.

Recently, selective state space models (S6) [11,12] have demonstrated efficient sequence modeling with linear-time complexity. By dynamically modulating SSM’s parameters based on the input, they can filter task-irrelevant patterns while retaining critical long-term information. This selective modulation property motivates revisiting LSTM gates from a state-space perspective, into which Kalman-inspired feedback can be injected, thereby endowing the gating mechanism with Kalman-optimal structural constraints.

In this work, we propose the Kalman-Optimal Selective Long-Term Memory (KOSLM) model, which establishes a context-aware feedback pathway that optimally balances memory retention and information updating, providing both theoretical interpretability and practical efficiency.

Our main contributions are as follows:

• State-space reformulation of LSTM: We formalize LSTM networks as input- and state-dependent state space models (SSMs), where each gate dynamically parameterizes the state-transition and input matrices. This framework provides a principled explanation of LSTM’s long-term memory behavior.
• Kalman-optimal selective gating: Inspired by the Kalman filter and selective SSMs, we introduce a Kalman-optimal selective mechanism in which the state-transition and input matrices are linearly modulated by a Kalman gain learned from the innovation term, establishing a feedback pathway that minimizes state estimation uncertainty.
• Applications to real-world forecasting: KOSLM consistently outperforms state-of-the-art baselines across long-term time series forecasting (LTSF) benchmarks in energy, finance, traffic, healthcare, and meteorology, achieving 10–30% lower mean squared error (MSE) and up to 2.5× faster inference compared with Mamba-2. In real-world Secondary Surveillance Radar (SSR) tracking under noisy and irregular sampling, KOSLM demonstrates strong robustness and generalization ability.

By bridging heuristic LSTM gating with principled Kalman-optimal estimation, KOSLM provides a robust, interpretable, and scalable framework for long-term sequence modeling, offering both methodological novelty and practical forecasting utility.

2. Background and Theory

2.1. LSTM Networks

RNNs model sequential data through recursive temporal computation. However, traditional RNNs often suffer from vanishing and exploding gradients when capturing long-term dependencies. The LSTM network [3] addresses this issue by introducing gating mechanisms that regulate the flow of information over time. The network structure of the LSTM neuron is shown in Figure 1.

Each LSTM unit maintains a cell state $C_t$ that carries long-term information and a hidden state $H_t$ that provides short-term representations. The cell state is updated according to:

$$C_t=F_t\odot C_{t-1}+I_t\odot {\tilde{C}}_t,$$

(1)

where $F_t$, $I_t$, and $O_t$ denote the forget, input, and output gates, respectively. These gates are nonlinear functions of the current input $x_t$ and the previous hidden state $H_{t-1}$. The final output is obtained as:

$$H_t=O_t\odot tanh\left(C_t\right).$$

(2)

This gating mechanism allows the model to selectively retain or discard information, mitigating gradient degradation. However, the gates are learned heuristically through data-driven optimization rather than derived from an explicit structural optimality constraint, making the LSTM sensitive to noise and unstable for long-term dependencies.

2.2. State Space Models

2.2.1. Selective State Space Models

S6 are a recent class of sequence models for deep learning that are broadly related to RNNs and classical SSMs. They are inspired by a particular system (Equation (3)) that maps a 1-dimensional function or sequence $x\left(t\right)\in \mathbb{R}\to y\left(t\right)\in \mathbb{R}$ through an implicit latent state $h\left(t\right)\in {\mathbb{R}}^N$:

$$\begin{array}{cc}\hfill h_t& =A{h}_{t-1}+B{x}_t,\hfill \end{array}$$

(3a)

$$\begin{array}{cc}\hfill y_t& =M{h}_t.\hfill \end{array}$$

(3b)

These models integrate the SSM described above into deep learning frameworks and introduce input-dependent selective mechanisms (see Appendix A for a detailed discussion), achieving Transformer-level modeling capability with linear computational complexity.

The theoretical connections among LSTM, KF, and SSM provide the foundation for constructing a unified Kalman-optimal selective memory framework.

2.2.2. Kalman Filter

The KF [6] is a classical instance of the state-space model, providing the minimum mean-square-error (MMSE) estimate of hidden system states under noisy observations. The system dynamics are expressed as:

$$\begin{array}{cc}\hfill h_t& =A{h}_{t-1}+w_t,w_t\sim \mathcal{N}(0,Q_t),\hfill \end{array}$$

(4a)

$$\begin{array}{cc}\hfill z_t& =M{h}_t+v_t,v_t\sim \mathcal{N}(0,R_t),\hfill \end{array}$$

(4b)

where A and M are the state transition and observation matrices; $w_t$ and $v_t$ denote process and observation noise with covariances $Q_t$ and $R_t$, respectively. Note that the observation $z_t$ can be regarded as the input $x_t$ in the SSMs.

At each time step, the KF performs two operations: prediction and update. The prediction step estimates the prior state:

$${\widehat{h}}_t^{-}=A{\widehat{h}}_{t-1},$$

(5)

while the update step refines this prediction using the observation $z_t$:

$$K_t={\widehat{P}}_t^{-}{M}^T{(M{\widehat{P}}_t^{-}{M}^T+R_t)}^{-1}$$

(6)

$${\widehat{h}}_t={\widehat{h}}_t^{-}+K_t(z_t-M{\widehat{h}}_t^{-})$$

(7)

Here, $K_t$ is the Kalman gain, which minimizes the posterior estimation error covariance. It determines how much new information (the innovation term $z_t-M{\widehat{h}}_t^{-}$) should be incorporated into the updated state. This principle of optimal selective information integration forms the theoretical foundation for the innovation-driven gating design proposed later. A detailed justification for interpreting the Kalman gain as a prototype of dynamic selectivity is provided in Appendix B.1

3. Proposed Method

3.1. Reformulating LSTM as a State-Space Model

Following the LSTM formulation in Equation (1), the LSTM cell can be equivalently reformulated as a time-varying SSM, where the cell state $C_t$ evolves under nonlinear, input- and state-dependent dynamics:

$$C_t=A_t{C}_{t-1}+B_t{z}_t,$$

(8)

Here, $z_t$ serves as the observation in the KF (corresponding to the input $x_t$ in the SSM and LSTM), and the matrices $A_t$ and $B_t$ are determined by the forget gate $F_t$ and input gate $I_t$, respectively. A detailed derivation of this LSTM-to-SSM reconstruction, including the mapping of gating mechanisms to state-space parameters, is provided in Appendix B.2.

3.2. Kalman-Optimal Selectivity via Innovation-Driven Gain

We introduce the innovation term from the KF:

$${\mathit{Innov}}_t=z_t-M_t{A}_{t-1}{C}_{t-1},$$

(9)

which measures the discrepancy between the observation input $z_t$ and the predicted state based on the previous cell state $C_{t-1}$, serving as a real-time correction signal between model prediction and actual measurement.

In classical KF, this innovation drives the computation of the Kalman gain $K_t$ (Equation (6)), which regulates the incorporation of new information and the retention of prior state during the update step. In KOSLM, rather than explicitly solving the Riccati recursion [13], we learn a functional mapping:

$$K_t=\varphi ({\mathit{Innov}}_t;{\theta }_{\varphi }),$$

(10)

where $\varphi (·)$ is a lightweight neural module implemented as a two-layer MLP with sigmoid activation, parameterized by ${\theta }_{\varphi }$. The use of the sigmoid ensures that the estimated gain $K_t\in (0,1)$, maintaining physical interpretability as an adaptive weighting coefficient and preventing numerical instability.

The learned gain dynamically regulates the trade-off between prior memory and new information, yielding a learnable yet principled mechanism for Kalman-optimal selectivity. The bounded output range of $K_t$ further stabilizes the state-space update and constrains divergence during long-horizon inference. Appendix B.3 provides the theoretical derivation demonstrating that the innovation term serves as a sufficient statistic for learning the Kalman gain. Appendix B.4 further presents controlled experiments confirming that the learned gain accurately approximates the oracle Kalman gain across various $(Q,R)$ regimes (Table A1). These results jointly establish the theoretical and empirical foundation for embedding Kalman-optimal selectivity into deep learning models.

We then define the state-space evolution as:

$$A_t=(I-K_t{M}_t)A,B_t=K_t,$$

(11)

where A is the base state transition matrix, serving as a learnable parameter of the model.

The KOSLM network architecture is illustrated in Figure 2, mirroring the classical Kalman update while maintaining differentiability and learnability.

3.3. Structural Overview of KOSLM

The KOSLM cell preserves the computational efficiency of a standard LSTM while embedding a Kalman-inspired feedback loop. At each timestep, the operations are:

• Initialization: The matrix A is initialized following S4D-Lin and for the real case is S4D-Real [14], which is based on the HIPPO theory [15]. These define the n-th element of A as $-{\displaystyle \frac{1}{2}}+ni$ and $-(n+1)$ respectively;
• Compute the innovation: ${\mathit{Innov}}_t=z_t-M_t{A}_{t-1}{C}_{t-1}$;
• Estimate the Kalman gain: $K_t=\varphi \left({\mathit{Innov}}_t\right)$;
• Update state transition matrices: $A_t=(I-K_t{M}_t)A$, $B_t=K_t$;
• Propagate the hidden state: $C_t=A_t{C}_{t-1}+B_t{z}_t$;
• Compute the output hidden representation: $H_t=M_t{C}_t$.

To further clarify the conceptual and structural differences between KOSLM and existing Kalman-based neural architectures, Table 1 summarizes a direct comparison with the KalmanNet [9].

3.4. Theoretical Interpretation

Under linear–Gaussian assumptions and with a sufficiently expressive mapping $\varphi$, the learned gain $K_t$ converges to the oracle Kalman solution. Consequently, KOSLM inherits the nonlinear expressive power of LSTM while achieving the minimum-variance estimation property of the Kalman filter in its linear regime. This leads to improved stability and robustness, particularly in long-horizon or noisy sequence modeling.

3.5. Practical Advantages

KOSLM offers several practical benefits:

• Robustness: The feedback structure mitigates error accumulation and improves performance under noise or distributional shifts.
• Efficiency: With only 0.24M parameters, KOSLM achieves up to 2.5× faster inference than Mamba-2, while maintaining competitive accuracy.
• Versatility: The model generalizes across diverse domains, from energy demand forecasting to radar-based trajectory tracking.

4. Results

This section provides a comprehensive evaluation of the proposed KOSLM model through large-scale long-term forecasting benchmarks (§Section 4.1), component ablation studies (§Section 4.2), efficiency assessments (§Section 4.3), and a real-world radar trajectory tracking case study (§Section 4.4). The experimental analyses collectively aim to validate both the predictive accuracy and robustness of KOSLM across diverse and noisy temporal conditions.

4.1. Main Experiments on Benchmark Datasets

To examine the capability of KOSLM in modeling long-range dependencies and maintaining stability over extended forecasting horizons, we conduct systematic experiments on nine widely used real-world datasets covering domains such as traffic flow, electrical load, exchange rate, meteorology, and epidemiology. The datasets vary in frequency, dimensionality, and temporal regularity, providing a comprehensive benchmark for assessing model generalization.

4.1.1. Dataset Details

We summarize the datasets used in this study as follows. Weather [16] contains 21 meteorological variables (e.g., temperature and humidity) recorded every 10 minutes throughout 2020. ETT (Electricity Transformer Temperature) [17] includes four subsets: two hourly-level datasets (ETTh1, ETTh2) and two 15-minute-level datasets (ETTm1, ETTm2). Electricity [18], derived from the UCI Machine Learning Repository, records hourly power consumption (kWh) of 321 clients from 2012 to 2014. Exchange [19] comprises daily exchange rates among eight countries. Traffic [20] consists of hourly road occupancy rates measured by 862 sensors on San Francisco Bay Area freeways from January 2015 to December 2016. Illness (ILI) dataset [21] tracks the weekly number of influenza-like illness patients in the United States. Table 2 summarizes the statistical properties of all nine benchmark datasets. All datasets are divided into training, validation, and test subsets with a ratio of 7:1:2.

4.1.2. Implementation Details

All models are trained using the Adam optimizer without weight decay. The learning rate is selected from $[1\times {10}^{-3},1\times {10}^{-2}]$ via grid search. Batch size is set to 32 by default, adjustable up to 256 depending on GPU memory. Training is performed for 15 epochs, and the checkpoint with the lowest validation loss is used for testing. Experiments are repeated five times, and average results are reported. All models adopt a 2-layer architecture with hidden dimension 64. We use PyTorch’s default weight initialization, and no additional regularization (dropout or gradient clipping) is applied. All experiments are implemented in PyTorch 2.1.0 with Python 3.11 on NVIDIA RTX 4090 GPUs. Random seeds for Python, NumPy, and PyTorch are fixed to ensure reproducibility.

The gain function $\varphi$ is implemented as a one-layer MLP with hidden dimension 64 and sigmoid activation, followed by a linear projection to the gain matrix $K_t$. The same $\varphi$ network is shared across all timesteps to ensure parameter consistency.

4.1.3. Experimental Setup

All experiments follow the evaluation protocol established in xLSTMTime [22], adopting prediction horizons of $T\in \{96,192,336,720\}$ for standard datasets and $T\in \{24,36,48,60\}$ for the weekly-sampled ILI dataset. We compare the proposed KOSLM with nine recent state-of-the-art baselines that represent diverse architectural paradigms, spanning state-space, recurrent, attention-based, and linear modeling frameworks:

• SSM-based: FiLM [23], S-Mamba [24];
• LSTM-based: xLSTMTime [22];
• Transformer-based: FEDformer [25], iTransformer [26], Crossformer [27];
• MLP/TCN-based: DLinear [28], PatchTST [29], TimeMixer [30].

This comprehensive selection enables a fair and systematic comparison across diverse sequence modeling paradigms.

4.1.4. Overall Performance

Table 3 and Table 4 report the long-term multivariate forecasting results on nine real-world datasets, evaluated by mean squared error (MSE) and mean absolute error (MAE). Across nearly all datasets and prediction horizons, the proposed KOSLM achieves the best or near-best performance, highlighting its superior generalization and robustness under diverse temporal dynamics.

• Consistent superiority across domains: KOSLM outperforms all competing baselines, particularly under complex and noisy datasets such as Traffic, Electricity, and ILI. In terms of average MSE reduction, KOSLM achieves relative improvements of +31.96% on Traffic, +13.37% on Electricity, +5.47% on Exchange, +11.26% on Weather, and +22.04% on ILI. Moreover, on the four ETT benchmarks (ETTh1/2, ETTm1/2), KOSLM yields steady improvements ranging from 4.03% to 27.46%, demonstrating strong adaptability to varying periodic and nonstationary patterns. These consistent gains verify that the Kalman-inspired selective updating mechanism effectively filters noise and dynamically adjusts to regime shifts, leading to stable forecasting accuracy over long horizons.
• Stable error distribution and reduced variance: The MSE–MAE gap of KOSLM remains narrower than that of other baselines, implying reduced large deviations and more concentrated prediction errors. This indicates more stable error behavior, which is crucial for long-horizon forecasting where cumulative drift often occurs. The innovation-driven Kalman gain estimation provides adaptive correction at each timestep, ensuring smooth and consistent prediction trajectories under uncertain dynamics.
• Strong scalability and generalization: KOSLM achieves leading performance not only on large-scale datasets (Traffic, Electricity) but also on small, noisy datasets (ILI), confirming robust generalization across different temporal resolutions and noise levels. Its consistent advantage over Transformer-based (e.g., iTransformer, FEDFormer, PatchTST), recurrent (e.g., xLSTMTime), and state-space models (e.g., S-Mamba, FiLM) demonstrates that the proposed Kalman-optimal selective mechanism provides an effective inductive bias for modeling long-term dependencies.
• Advantage over LSTM-based architectures: Compared with advanced LSTM-based models such as xLSTMTime, KOSLM achieves consistently better results across nearly all datasets and horizons. This verifies that replacing heuristic gates with Kalman-optimal selective gating enhances memory retention and update stability. While xLSTMTime alleviates gradient decay via hierarchical memory, KOSLM further refines state updates through innovation-driven gain estimation, thereby achieving a more principled and stable information flow.

To further demonstrate the long-horizon stability of the proposed model, we compare KOSLM with the recent S-Mamba [24], a state-of-the-art state space model that represents the latest advancement in efficient sequence modeling. Figure 3 presents the forecasting trajectories at the long horizon ($T=720$) across five representative datasets. KOSLM maintains accurate trend alignment and amplitude consistency with the ground truth, showing particularly superior convergence behavior on smoother and more stationary datasets such as Weather and Exchange, where the Kalman gain adaptation stabilizes long-term predictions. In contrast, S-Mamba exhibits slight temporal lag and amplitude attenuation under extended forecasting conditions. These results visually confirm the advantage of the proposed Kalman-based feedback selectivity in preserving long-term temporal fidelity.

4.2. Ablation Study

To further analyze the effectiveness of each proposed component, we conduct ablation experiments on four widely used long-term time-series forecasting datasets (ETTm1, ETTh1, Traffic, and Exchange) with a prediction length of $L=720$, following the data preprocessing described in Section 4.1.1. All model variants are trained and tested following the default implementation settings described in Section 4.1.2, and the results are averaged over five runs to ensure statistical reliability.

4.2.1. Structural Ablation.

We first evaluate the contribution of the Kalman-based structure by comparing three variants: (i) Full (KOSLM): the complete model where $K_t=\varphi \left(\mathrm{Innov}\right)$ and $A_t=(I-K_t{M}_t)A,B_t=K_t$; (ii) No-Gain: $\varphi$ receives the innovation but directly outputs $(A_t,B_t)$, removing the explicit Kalman gain variable; (iii) No-Innov: $\varphi$ receives the standard network input (e.g., $[x_t;H_{t-1}]$) instead of the innovation, while $K_t$ is still mapped to $(A_t,B_t)$ via the Kalman form.

As shown in Table 5, removing either the gain computation or the innovation input consistently leads to higher errors across all datasets. For instance, on ETTm1, the MSE increases by 32.9% when the gain path is removed, confirming that both the innovation statistic and Kalman gain are essential for stable long-horizon prediction. This finding aligns with the Kalman filtering principle that the innovation serves as a sufficient statistic for state correction.

4.2.2. Capacity Ablation

To verify that the performance gains of KOSLM mainly stem from the Kalman-optimal structural design rather than the network capacity, we conduct a capacity ablation study. We evaluate the $\varphi$ network under four progressively larger configurations: (i) Linear: a single linear layer with input dimension equal to the number of input features and output dimension 64, without any activation; (ii) SmallMLP: a two-layer MLP with 128 hidden units and sigmoid activation; (iii) MediumMLP: a three-layer MLP with 256 hidden units per layer and Rsigmoid activation; (iv) HighCap: a four-layer MLP with 512 hidden units per layer and sigmoid activation.

All models are trained under identical settings with five independent random seeds, and the results are reported as mean ± standard deviation. This allows us to evaluate both performance trends and statistical reliability. From Table 6, it is evident that increasing the $\varphi$ network capacity beyond SmallMLP does not consistently improve MSE or MAE; in some cases, the error even increases slightly. The small MediumMLP improvements (ETTm1) or slight deterioration (ETTh1, Traffic, Exchange) indicate that performance gains are dominated by the Kalman-optimal structural design rather than network depth or over-parameterization. This confirms the effectiveness of the innovation-driven gain estimation in KOSLM for stable long-horizon prediction.

4.3. Efficiency Benchmark

To evaluate the computational efficiency of KOSLM, we benchmark it against representative sequence modeling baselines, including Transformer, Mamba-2, and LSTM, across four key metrics: runtime scalability, end-to-end throughput, memory footprint, and parameter count. All experiments are conducted using PyTorch 2.1.0 with torch.compile optimization on a single NVIDIA RTX 4090 GPU.

4.3.1. Runtime Scalability

We assess runtime performance on both a controlled synthetic setup and the ETTm1 dataset. As illustrated in Figure 4, KOSLM exhibits near-linear growth in inference time with increasing sequence length L. For sequences longer than $L=2\mathrm{k}$, KOSLM surpasses optimized Transformer implementations (FlashAttention [31]) and achieves up to $2.5\times$ faster execution than the fused-kernel version of Mamba-2 [24] at $L>4\mathrm{k}$. Compared with a standard PyTorch LSTM, KOSLM achieves approximately $1.3$–$1.9\times$ speedup across the tested sequence lengths, demonstrating robust scalability for long sequences.

4.3.2. Throughput Analysis.

Figure 5 reports the end-to-end inference throughput measured in million tokens/s (M tokens/s). KOSLM maintains consistently high throughput across all tested sequence lengths, achieving up to $1.9\times$ higher throughput than LSTM and $2.7\times$ higher than Mamba-2 in the 4K–8K regime. The performance remains stable for even longer sequences, highlighting the model’s suitability for extended temporal dependencies.

4.3.3. Memory Footprint.

Table 7 reports GPU memory usage during training for varying batch sizes with input length 2048 and models containing approximately 0.24M parameters. KOSLM demonstrates favorable memory efficiency relative to similarly sized Mamba-2 implementations and remains comparable to a standard LSTM with roughly ten times more parameters. This efficiency results from KOSLM’s compact Kalman-optimal selective mechanism, which dynamically modulates state transitions with minimal parameter overhead.

4.3.4. Model Size.

Table 8 summarizes the parameter counts. KOSLM contains only 0.24M parameters, roughly 4.5% of the Transformer and 11% of the LSTM baselines, while achieving comparable or superior forecasting performance. The compact design emphasizes KOSLM’s suitability for deployment in resource-constrained environments.

This efficiency stems directly from the Kalman-optimal selective mechanism, which adaptively regulates the state update through innovation-driven gain modulation. Such a formulation not only reduces redundant computation but also provides a principled pathway for scaling to long sequences.

4.4. Real-world Application: SSR Target Trajectory Tracking

To further evaluate the robustness and practical generalizability of KOSLM under real-world noisy conditions, we conducted a real-world experiment on secondary surveillance radar (SSR) target trajectory tracking. SSR is a ground-based air traffic surveillance system where aircraft respond to radar interrogations with encoded transponder signals, generating sparse range–azimuth sequences known as raw SSR plots. These observation sequences present three major challenges for sequential modeling:

• High stochastic noise: Measurement noise leads to random fluctuations in the estimated positions;
• Irregular sampling: Aircraft maneuvers and radar scan intervals result in uneven temporal spacing;
• Correlated anomalies: Spurious echoes or missing detections introduce discontinuities in the trajectories.

These characteristics make SSR a natural yet challenging testbed for assessing model stability, noise resilience, and temporal consistency. KOSLM addresses these challenges by integrating context-aware selective state updates with learnable dynamics under the Kalman optimality principle, enabling robust filtering, smoothing, and extrapolation in partially observed environments.

4.4.1. Experimental Setup

To achieve a balance between realism and experimental controllability, we adopted a semi-physical simulation approach. Training data were derived from Automatic Dependent Surveillance–Broadcast (ADS-B) flight tracks obtained from the OpenSky Network [32]. Controlled Gaussian noise with an SNR of 33 dB was added to emulate SSR observation uncertainty. For evaluation, we deployed KOSLM on real SSR radar data collected from an operational ground-based system (peak transmit power 6 kW), capturing eight live air-traffic targets under operational conditions. Detailed data acquisition and preprocessing procedures are provided in Appendix C.

Unlike the main benchmarking experiments on standardized long-term time series forecasting (LTSF) datasets, this case study aims to demonstrate practical applicability and operational robustness. Due to irregular sampling, sparse observations, and high stochastic noise, conventional quantitative metrics such as MSE or MAE are not meaningful. Therefore, we focus on qualitative trajectory visualizations to illustrate model performance in realistic conditions.

4.4.2. Results and Analysis

Figure 6 presents representative trajectories of eight air-traffic targets, comparing classical KF algorithm, Transformer, Mamba, and KOSLM. The classical KF algorithm exhibits locally inaccurate and jagged trajectories due to its fixed linear dynamics, which cannot adapt to irregular sampling or abrupt maneuvers. The Transformer captures general trends but produces fragmented and temporally inconsistent tracks under sparse and noisy observations. Mamba improves noise robustness but still demonstrates local instability during complex maneuvers.

In contrast, KOSLM generates smooth, coherent, and temporally consistent trajectories that closely follow the true flight paths, highlighting its ability to handle high stochastic noise, irregular sampling, and correlated anomalies. This robustness stems from the innovation-driven Kalman gain and context-aware selective state updates, which allow KOSLM to adapt dynamically to non-stationary motion patterns.

5. Conclusions

This study addressed a fundamental limitation of recurrent architectures such as LSTM, whose heuristically designed gates lack structural constraints for optimality, leading to instability and information decay over long sequences. To overcome this issue, we proposed the Kalman-Optimal Selective Long-Term Memory Network (KOSLM), which reconstrues LSTM as a nonlinear, input- and state-dependent state-space model, and integrates an innovation-driven Kalman-optimal gain path for principled information selection. This formulation unifies LSTM gating, selective state-space modeling, and Kalman filtering into a single theoretically grounded recurrent framework.

Extensive experiments demonstrate that KOSLM achieves state-of-the-art performance on long-term forecasting benchmarks, while ablation studies confirm the essential role of the innovation statistic and Kalman-form gain in stabilizing long-horizon modeling. Moreover, validation on real-world SSR trajectory tracking highlights its robustness under noisy and non-stationary conditions. In summary, embedding Kalman-optimal principles into deep recurrent networks provides both theoretical insights and practical benefits for robust long-term sequence modeling. Future work will focus on extending KOSLM beyond Gaussian assumptions and applying it to multimodal and cross-domain time-series scenarios.

Appendix B.1. Kalman Gain as a Prototype for Dynamic Selectivity

In classical filtering theory, the Kalman gain $K\left(t\right)$ acts as a dynamic weighting factor that determines how prior state estimates and new observations are combined to produce a new state [6]. It is formally derived as the solution to the Riccati differential equation, and its value depends on the evolving uncertainty in the internal system state (captured by the prior covariance $P\left(t\right)$) and the noise characteristics of the observation process (captured by the measurement covariance R) [41]. This time-varying gain governs the extent to which incoming measurements correct the state estimate, ensuring minimum mean-squared error under Gaussian noise assumptions.

This correction process is essentially a trade-off between the historical state information of the system and the incoming input information. In the context of deep learning, we interpret the observation as the current input sequence, and the prior state estimate as the latent representation of the system’s history. From this perspective, the Kalman gain plays a role analogous to a dynamic selection factor that balances the contribution of contextual knowledge (from the model state) and content information (from the input) in generating the updated representation. Therefore, the Kalman gain can be viewed as a principled prototype for content-aware and context-sensitive information selection. This perspective justifies the broader use of dynamic selection strategies in sequence models, especially when aiming to balance stability and adaptability in long-range modeling.

Appendix B.2. LSTM-to-SSM Reconstruction

To endow LSTM networks with structured modeling semantics, in this section we reconstruct them as a form of nonlinear time-varying state-space model ((see Equation (3a)). Specifically, we identify that the interactions among the forget gate, input gate, output gate, input signal, and cell state in LSTMs essentially constitute an input-dependent and state-aware state transition–observation process.

As illustrated in Figure 1, the gating mechanisms and the cell state $C_t$ in LSTM establish a pathway for memory propagation across time. The cell state $C_t$ is the core component that preserves long-term memory across the temporal dimension and models long-range dependencies. In terms of representational role, it is both semantically and structurally equivalent to the latent state $h_t$ in SSMs (see Equation (3a)). Therefore, we take $C_t$ as the central axis and analyze its information propagation process.

The forget gate $F_t$ regulates the degree to which short-term memory from cell state $C_{t-1}$ is discarded. It is determined jointly by the current input $x_t$ and the previous hidden state $H_{t-1}$, thereby embodying an input- and state-dependent transition process:

$$A_t\triangleq F_t=\sigma (W_f{X}_t+b_f),$$

(A1)

where $X_t=[x_t;H_{t-1}]$ denotes the concatenated input. The corresponding retained term is $A_t{C}_{t-1}$.

The pathway for new information input in LSTM consists of two steps:

• The candidate cell state ${\tilde{C}}_t$ can be interpreted as a differentiable nonlinear mapping from the joint input $X_t$:

  $$z_t\triangleq {\tilde{C}}_t=tanh(W_c{X}_t+b_c),$$

  (A2)

  where the linear transformation $W_c{X}_t+b_c$ fuses the external input $x_t$ and the hidden state $H_{t-1}$ into the observation space, with $tanh(·)$ introducing nonlinearity to produce an intermediate representation $z_t$. We treat this representation as the observation input at time t under the SSM.1
• The interaction between the input gate $I_t$ and the candidate state ${\tilde{C}}_t$ establishes a structured pathway for injecting new information into the state dynamics, analogous to the excitation of state evolution by external inputs in SSMs.

  $$B_t{z}_t\triangleq I_t\odot {\tilde{C}}_t=\sigma (W_i{X}_t+b_i)\odot tanh(W_c{X}_t+b_c),$$

  (A3)

  where $B_t$ serves as the input matrix, structurally equivalent to the input gate $I_t$.

Consequently, the update process of the LSTM cell state $C_t$ at time t (Eq.1) can be rewritten in the form of a state-space transition:

$$C_t=A_t{C}_{t-1}+B_t{z}_t,$$

(A4)

where $A_t$ and $B_t$ are realized through the forget and input gates driven by the joint input $X_t$, thereby endowing the model with the ability to selectively forget or memorize long-term information in an input-dependent and context-aware manner.

Finally, the output pathway of LSTM is given by

$$H_t=O_t\odot tanh\left(C_t\right),$$

(A5)

which is formally equivalent to the observation equation of a classical SSM:

$$y_t=M_t{C}_t,$$

(A6)

where $M_t$ is formed by the output gate $O_t$ together with the nonlinear transformation $tanh(·)$.

The above reconstruction shows that the gating mechanism in LSTM can be interpreted as a class of nonlinear, input- and state-dependent SSMs. This perspective establishes a valid unification of LSTM gating mechanisms and state-space models, and reveals that LSTM’s long-term memory capability originates from the structured realization of both state-space modeling and efficient information selection. Building on this understanding, we further introduce the Kalman gain into this state-space structure, imposing a structural constraint that minimizes the uncertainty of state estimation errors.

Appendix B.3. Kalman-Optimal Selective Mechanism

Recent studies on time series modeling have proposed a class of selective mechanisms based on SSMs, among which the most representative work is S6 [11]. Its core idea points out:

“One method of incorporating a selection mechanism into models is by letting their parameters that affect interactions along the sequence (e.g., the recurrent dynamics of an RNN or the convolution kernel of a CNN) be input-dependent.”

Inspired by this, we design a selective mechanism based on Kalman-optimal state estimation within the SSM (Equation (3)). Unlike the input-dependent selection mechanism represented by S6, we make the key parameters ($A_t$, $B_t$), which control how the model selectively propagates or forgets information along the sequence dimension, depend on the innovation term: the deviation between the observation input and the prior state prediction in the observation space, denoted as $\mathit{Innov}$ in this paper. This method constructs a learnable selection path that integrates observational inputs and latent state feedback, with the optimization objective of minimizing the uncertainty in state estimation errors.

• Learnable Gain from Innovation

In the KF algorithm, the closed-form solution of the Kalman gain $K_t$ (Equation (6)) is obtained by minimizing the covariance of the state estimation error [6]. Here, the observation matrix $M_t$ and the observation noise covariance $R_t$ are considered known system priors, usually derived from physical modeling or domain knowledge. Therefore, the dynamics of $K_t$ entirely stem from the statistical properties of the prior estimation error $e_t^{-}=h_t-{\widehat{h}}_t^{-}$. Under the linear observation model (4b), $e_t^{-}$ is mapped into the observation space, yielding the following representation of the innovation term:

$$\mathit{Innov}=z_t-M_t{\widehat{h}}_t^{-}=M_t\left(h_t-{\widehat{h}}_t^{-}\right)+v_t,$$

(A7)

This equation shows that $\mathit{Innov}$ is a linear mapping of $e_t^{-}$, which, although perturbed by additive observation noise $v_t$, still retains the full uncertainty information of $e_t^{-}$. Thus, $\mathit{Innov}$ can serve as a sufficient statistic for estimating $K_t$, providing a theoretical basis for directly estimating $K_t$ from the innovation term. The estimation process is defined as:

$$K_t=\varphi ({\mathit{Innov}}_t;{\theta }_{\varphi }),$$

(A8)

where $\varphi (·)$ is a differentiable nonlinear function, parameterized by a neural network and trained via gradient descent. Theoretical soundness of this formulation is confirmed by a controlled experiment (Appendix B.4), which demonstrates that $\varphi (·)$ can reliably recover the oracle Kalman gain from the innovation statistics with negligible estimation error.

• Optimal Selectivity via Gain

By substituting the KF prediction process (Equation (5)) into the state update equation (Equation (7)), the two processes can be expressed as a nonlinear time-varying SSM form (Equation (3a)):

$$\begin{array}{cc}\hfill {\widehat{h}}_t& =A_t{\widehat{h}}_{t-1}+B_t{z}_t,A_t=(I-K_t{M}_t)A,B_t=K_t,\hfill \end{array}$$

(A9)

Within this framework, parameters $A_t$ and $B_t$ are linearly modulated by $K_t$ driven by the innovation term, yielding a context-aware Kalman-optimal selective path that minimizes state estimation uncertainty and imposes structural optimality constraints on LSTM.

Appendix B.4. Empirical Validation of Innovation-Driven Kalman Gain Learning

To empirically validate the theoretical claim that the Kalman gain can be dynamically inferred from the innovation term, we design a controlled linear–Gaussian synthetic experiment where the optimal gain has a closed-form solution. This allows us to directly compare the learned gain against the ground-truth Kalman gain and quantify their alignment over time.

• Experimental Design

We consider a one-dimensional linear dynamical system as in Equation (4). The analytical Kalman gain $K_t$ is computed from the standard Riccati recursion and serves as the oracle reference. We design three model variants for comparison:

(i)

Oracle-KF: Standard Kalman filter using the known $(A,Q,R,M)$ parameters to compute $K_t$.

(ii)

Supervised-$\varphi$: A small MLP $\varphi (·)$ takes the innovation$inno{v}_t=z_t-M{\widehat{h}}_t^{-}$ as input and predicts ${\widehat{K}}_t=\varphi \left(inno{v}_t\right)$, trained by minimizing $\mathcal{L}=|{\widehat{K}}_t-K_t{|}^2$ over all timesteps. During training, the predicted prior state ${\widehat{h}}_t^{-}$ is provided by the Oracle-KF to isolate the learning of the innovation-to-gain mapping. This tests whether the innovation term contains sufficient information to recover the oracle gain.

(iii)

End-to-End: The same MLP is trained to directly map from $(z_t,{\widehat{h}}_t^{-})$ to $K_t$ without explicitly constructing the innovation term, serving as an ablation to assess the importance of innovation-driven modeling.

• Setup and Metrics

We simulate $10,000$ trajectories of length $T=100$ with fixed system parameters $(A=0.9,M=1)$. To test robustness under different noise regimes, we consider four parameter settings by varying process noise Q and observation noise R: (i) $Q=5,R=0.1$; (ii) $Q=5,R=0.5$; (iii) $Q=10,R=0.1$; (iv) $Q=100,R=0.5$. The supervised models are trained for 200 epochs using Adam optimizer with a learning rate ${10}^{-3}$. Performance is evaluated by:

• MSE of Gain:$\mathrm{MSE}={\displaystyle \frac{1}{T}}{\sum }_t{|K_t-{\widehat{K}}_t|}^2$, measuring how well the learned gain matches the oracle trajectory.
• State Estimation RMSE: Root-mean-square error between estimated and true state trajectories, where the state estimates are produced by running a Kalman update step with the learned ${\widehat{K}}_t$, verifying that accurate gain learning improves filtering quality.

Table A1. Quantitative Results of Kalman Gain Learning. Supervised-$\varphi$ achieves orders-of-magnitude lower gain MSE and consistently better state estimation RMSE across all noise regimes compared to End-to-End learning, highlighting the critical role of explicitly modeling the innovation term in recovering accurate Kalman gain dynamics.

(Q, R)	Supervised-$\mathit{\varphi }$		End-to-End
	MSE	RMSE	MSE	RMSE
(5, 0.1)	$1.3\times {10}^{-6}$	0.317	$3.5\times {10}^{-3}$	0.368
(5, 0.5)	$2.9\times {10}^{-5}$	0.684	$1.8\times {10}^{-3}$	0.694
(10, 0.1)	$1.2\times {10}^{-6}$	0.216	$1.3\times {10}^{-3}$	0.341
(100, 0.5)	$2.2\times {10}^{-7}$	0.310	$6.8\times {10}^{-4}$	0.747

Table A1. Quantitative Results of Kalman Gain Learning. Supervised-$\varphi$ achieves orders-of-magnitude lower gain MSE and consistently better state estimation RMSE across all noise regimes compared to End-to-End learning, highlighting the critical role of explicitly modeling the innovation term in recovering accurate Kalman gain dynamics.

(Q, R)	Supervised-$\mathit{\varphi }$		End-to-End
	MSE	RMSE	MSE	RMSE
(5, 0.1)	$1.3\times {10}^{-6}$	0.317	$3.5\times {10}^{-3}$	0.368
(5, 0.5)	$2.9\times {10}^{-5}$	0.684	$1.8\times {10}^{-3}$	0.694
(10, 0.1)	$1.2\times {10}^{-6}$	0.216	$1.3\times {10}^{-3}$	0.341
(100, 0.5)	$2.2\times {10}^{-7}$	0.310	$6.8\times {10}^{-4}$	0.747

• Results and Analysis

Figure A1 presents the learned Kalman gain trajectories across all considered noise regimes. Across every $(Q,R)$ configuration, the Supervised-$\varphi$ model closely matches the oracle Kalman gain $K_t$, producing stable and accurate trajectories throughout both transient and steady phases. In contrast, the End-to-End model produces a noisier and slightly biased gain curve, indicating that omitting the explicit innovation term fails to accurately capture the gain dynamics. This qualitative observation is quantitatively confirmed in Table A1: Supervised-$\varphi$ achieves up to four orders-of-magnitude lower gain MSE and reduces state-estimation RMSE by $4.5\%-32.8\%$ under all $(Q,R)$ noise settings. These results demonstrate that the innovation term is indeed sufficient for inferring the optimal Kalman gain and that explicitly modeling innovation-driven selectivity yields more stable and accurate state estimation. Together, the figure and table provide strong empirical support for our theoretical claim that innovation-guided gain learning constitutes a principled and robust mechanism for selective state updates.

Figure A1. Kalman gain learning across different noise regimes. Top-left: $(Q=100,R=0.5)$; top-right: $(Q=5,R=0.1)$; bottom-left: $(Q=5,R=0.5)$; bottom-right: $(Q=10,R=0.1)$. Supervised-$\varphi$ closely matches the oracle $K_t$ in all cases, while End-to-End trajectories remain noisy and biased. These results confirm that innovation-driven modeling provides stable and accurate gain learning across both low- and high-noise conditions.

Figure A1. Kalman gain learning across different noise regimes. Top-left: $(Q=100,R=0.5)$; top-right: $(Q=5,R=0.1)$; bottom-left: $(Q=5,R=0.5)$; bottom-right: $(Q=10,R=0.1)$. Supervised-$\varphi$ closely matches the oracle $K_t$ in all cases, while End-to-End trajectories remain noisy and biased. These results confirm that innovation-driven modeling provides stable and accurate gain learning across both low- and high-noise conditions.

Appendix C.1. Data Acquisition

The semi-physical training dataset is constructed using ADS-B logs from the OpenSky Network, recorded on June 15, 2025, between 10:00 and 10:15 AM. The data cover a 300 km region surrounding Tokyo Haneda, Narita, and Incheon Airports. Each log provides updates every 5 seconds for 503 aircraft, including longitude, latitude, altitude, and timestamp information. A detailed statistical summary of the dataset is provided in Table A2.

To emulate realistic Secondary Surveillance Radar (SSR) observation noise, zero-mean Gaussian perturbations are applied to each measurement with a signal-to-noise ratio (SNR) of 33 dB, formulated as

$$X_i=X_i+\mathcal{N}\left(0,{\displaystyle \frac{{\sigma }_i^2}{{10}^{\frac{33}{10}}}}\right),$$

where ${\sigma }_i^2$ represents the empirical variance of the i-th feature. This noise configuration reflects typical radar tracking uncertainty under moderate atmospheric interference.

Table A2. Summary of the Semi-physical ADS-B Dataset Used for Training. The dataset consists of 5-second updates for 503 aircraft collected within a 15-minute window around major East Asian airports. Each sample includes four state variables: longitude, latitude, altitude, and timestamp.

Targets	Update Frequency	# Features	Total Samples	Time Span
503	5 seconds	4	166,110	2025–06–15, 10:00–10:15

Table A2. Summary of the Semi-physical ADS-B Dataset Used for Training. The dataset consists of 5-second updates for 503 aircraft collected within a 15-minute window around major East Asian airports. Each sample includes four state variables: longitude, latitude, altitude, and timestamp.

Targets	Update Frequency	# Features	Total Samples	Time Span
503	5 seconds	4	166,110	2025–06–15, 10:00–10:15

Appendix C.2. Normalization and Reverse Transformation

All input data are standardized to zero mean and unit variance:

$$y={\displaystyle \frac{x-\overline{x}}{\sigma }},\widehat{x}=\sigma \widehat{y}+\overline{x}.$$

Appendix C.3. Field Data Collection

For real-world evaluation, we use raw SSR plots collected by a field-deployed radar with a peak transmit power of 6 kW. These sequences contain irregular sampling, strong noise, and missing returns, providing a rigorous test of model robustness.

Figure A2. SSR Trajectory Modeling Pipeline. The upper branch shows training using ADS-B data with noise simulation and normalization; the lower branch shows testing on real raw SSR plots using trained models, followed by trajectory prediction and comparison.

Figure A2. SSR Trajectory Modeling Pipeline. The upper branch shows training using ADS-B data with noise simulation and normalization; the lower branch shows testing on real raw SSR plots using trained models, followed by trajectory prediction and comparison.

Appendix C.4. Processing Workflow

The overall pipeline—from semi-physical data generation to field testing—is summarized in Figure A2, where the upper branch denotes the training stage on noisy ADS-B data, and the lower branch represents inference on real SSR plots. This workflow ensures consistency between semi-physical training data and real-world SSR testing scenarios.