A Kalman Filter-Based Localization Calibration Method Optimized by Reinforcement Learning and Information Matrix Fusion

1. Introduction

With the rapid development of intelligent unmanned systems, high-precision positioning technology for unmanned clusters has become a significant challenge in its application. In dynamic scenarios, single sensors like Global Positioning System(GPS) often suffer from signal blockage and multipath effects, leading to degraded accuracy [1,2], while Inertial Measurement Unit (IMU) accumulates errors over time and cannot support long-term localization independently [3]. Therefore, Kalman Filter (KF) [4]-based multi-sensor fusion has become a mainstream solution, offering optimal estimation through a state-space model and theoretically enabling error minimization and calibration. However, traditional Kalman Filter relies on fixed parameters [5], leading to model mismatch in scenarios with motion state transitions (e.g., sudden acceleration, complex terrains) or time-varying sensor noise, which degrades localization accuracy—particularly in multi-trajectory cooperative scenarios.

Existing methods suffer from insufficient robustness in filter parameters and inefficient multi-source fusion. For example, process noise parameters in KF are typically empirically set and cannot dynamically adapt to environmental changes, causing the KF gain matrix to degrade in high-dynamic scenarios and resulting in drift. Traditional trajectory fusion also fails to fully exploit inter-trajectory correlations, such as varying covariance structures among nodes in a sensor network, making static weighting suboptimal. Reinforcement Learning, as a machine learning paradigm that optimizes decision-making through environment interaction [6], offers a new path forward. Its trial-and-error adaptation breaks the static limitations of traditional methods.

Therefore, this study proposes a Kalman Filter localization calibration method based on reinforcement learning optimization and information matrix fusion. An Actor-Critic network architecture [7] is constructed, where a Gated Recurrent Unit (GRU) [8] captures temporal dependencies of sensor data, and an attention mechanism [9] focuses on key state features to adaptively generate KF parameter adjustment factors. These factors dynamically regulate the state covariance matrix. Additionally, we introduce a multi-trajectory information matrix fusion algorithm that aggregates inverse covariance matrices to form a layered uncertainty calibration mechanism.

This paper is organized as follows: Section 2 reviews existing calibration methods; Section 3 details the algorithm, including the network architecture, dynamic adjustment mechanism for state covariance, and the derivation of the fusion algorithm based on information matrix inversion. Section 4 presents experimental design and results. Section 5 concludes the study and discusses future work and applications.

2. Related Work

At present, multi-sensor fusion has become the mainstream approach for localization calibration in UAV swarms. Among them, GPS/IMU integrated navigation systems are widely used due to their complementary advantages [10,11]. The KF, as a classical algorithm, performs optimal recursive estimation for linear systems. To address the significant errors introduced when using KF in nonlinear scenarios [12], several nonlinear filtering variants have been developed, including the Extended Kalman Filter (EKF) [13], Unscented Kalman Filter (UKF) [14], and Particle Filter (PF) [15]. Moreover, to improve filter stability in high-dynamic environments, adaptive variants such as the Adaptive Kalman Filter (AKF) [16] and Federated Kalman Filter (FKF) [17] have been proposed. These evolutionary filtering methods provide theoretical support for achieving high-precision localization in UAV swarms operating under complex environments, aggressive maneuvers, and sensor failures.

In the development of integrated navigation technologies, machine learning-based adaptive calibration methods have gradually emerged in recent years with the advancement of deep learning and reinforcement learning [18,19]. These methods leverage historical data to learn complex error patterns and achieve high-precision calibration. For example, Wei et al. [20] proposed a navigation method that integrates random forest regression with adaptive Kalman filtering to improve accuracy under limited data availability. Liu et al. [21] proposed an integrated navigation filtering algorithm assisted by Radial Basis Function(RBF) neural network, aiming to improve the navigation accuracy of the system in the case of GPS signal loss. Other studies [22] have combined deep learning with extended state Kalman filtering (ES-EKF), using Long Short-Term Memory (LSTM) [23] networks to capture temporal dependencies in sensor data and Convolutional Neural Network (CNN) [24] to extract spatial features. Furthermore, a reinforcement learning-based adaptive Kalman filtering algorithm (RL-AKF) [25] has been proposed, which adaptively estimates the process noise covariance using reinforcement learning techniques. In the Kalman filter framework, the state covariance matrix plays a central role in quantifying uncertainty and directly affects the computation of the Kalman gain. However, current methods still largely rely on traditional techniques for its adaptive adjustment. Therefore, this paper proposes a reinforcement learning-based approach using an Actor-Critic architecture to dynamically adjust the state covariance matrix.

Meanwhile, in the area of multi-trajectory cooperative calibration, inter-platform information exchange is a critical component for achieving high-precision collaboration. Existing mainstream methods focus on fusion strategies for heterogeneous multi-source data. For example, in cooperative localization of UAV swarms, platforms can share position and inertial data; Ultra-Wideband (UWB) ranging technology is used to acquire relative distance constraints [26]; Angle-of-Arrival (AOA) methods employ antenna arrays to determine signal directions [27], forming joint constraints in distance and angle; and vision sensors can be used to capture environmental features or visual markers of other UAVs, with visual SLAM algorithms employed to extract feature correspondences and construct relative pose estimates [28]. However, most existing approaches lack effective mechanisms for quantifying and fusing trajectory uncertainties, particularly in dynamically aggregating the covariance matrices of individual platform trajectories. Therefore, this study proposes a cooperative calibration algorithm based on multi-trajectory information matrix fusion. This approach enables a more precise characterization of uncertainty distribution across trajectories in dynamic environments and provides a novel solution for high-precision cooperative localization of UAV swarms in complex scenarios.

3. Methods

To address the challenges of trajectory parameter uncertainty quantification and dynamic calibration in cooperative localization of UAV swarms, this study proposes a collaborative localization framework that integrates reinforcement learning with information matrix fusion. A dual closed-loop architecture—“single-trajectory dynamic filtering and multi-trajectory cooperative correction”—is adopted (as shown in Figure 1). First, an Actor-Critic reinforcement learning network is used to adaptively optimize the state covariance matrix in the Kalman filter for each individual trajectory. Then, an information matrix fusion strategy is applied, in which each trajectory’s covariance matrix is transformed into its corresponding information matrix. By aggregating these matrices, the framework fuses the credibility of multi-source trajectory estimates and suppresses error propagation from individual platforms. The overall algorithm consists of two key components: reinforcement learning-driven filter parameter optimization and multi-trajectory information matrix fusion.

3.1. Reinforcement Learning-Driven Filter Parameter Optimization

In the state update process of the Kalman filter, this study employs a reinforcement learning mechanism to dynamically adjust the state covariance matrix.

We first define the system state vector $x_k={\left[\begin{array}{cccc}{p}_k& v_k& q_k& a_k\end{array}\right]}^T$ , including the platform’s position $p_k$, velocity $v_k$, quaternion $q_k$, and acceleration $a_k$. The state update follows:

$$x_{k+1}=f(x_k,u_k)+w_k$$

(1)

$$P_{k+1}=F_k{P}_k{F_k}^T+Q_k$$

(2)

where $x_{k+1}$ and $P_{k+1}$ are the predicted state vector and covariance matrix at time step k+1, $w_k$ is the process noise with covariance $Q_k$, and F is the Jacobian of the state transition function $f(·)$. The Jacobian matrix F of the state transition function $f(·)$ includes the dynamic coupling terms among position, velocity, and acceleration, as well as quaternion updates, where $\Omega \left(\omega \right)=\left[\begin{array}{cccc}0& -{\omega }_x& -{\omega }_y& -{\omega }_z\\ {\omega }_x& 0& {\omega }_z& -{\omega }_y\\ {\omega }_y& -{\omega }_z& 0& {\omega }_x\\ {\omega }_z& {\omega }_y& -{\omega }_x& 0\end{array}\right]$ is an antisymmetric matrix constructed from angular velocity:

$$F=\left[\begin{array}{cccc}{I}_3& dt·I_3& 0_{3\times 4}& 0.5·d{t}^2·I_3\\ 0_{3\times 3}& I_3& 0_{3\times 4}& dt·I_3\\ 0_{4\times 3}& 0_{4\times 3}& F_q& 0_{4\times 3}\\ 0_{3\times 3}& 0_{3\times 3}& 0_{3\times 4}& I_3\end{array}\right]$$

(3)

$$F_q=\left\{\begin{array}{c}{I}_4+{\displaystyle \frac{dt}{2}}·\Omega \left(\omega \right),\left|\right|\omega \left|\right|\ll 1\\ cos\left({\displaystyle \frac{\left|\right|\omega \left|\right|·dt}{2}}\right)I_4+{\displaystyle \frac{sin\left(\frac{\left|\right|\omega \left|\right|·dt}{2}\right)}{\left|\right|\omega \left|\right|}}\Omega \left(\omega \right),others\end{array}\right.$$

(4)

The observation model is given by:

$$y_k=z_k-H{x}_k$$

(5)

where $z_k$ is the actual measurement and $H=\left[\begin{array}{cccc}{I}_3& 0_3& 0_4& 0_3\end{array}\right]$ is the observation matrix.

Then calculate the Kalman gain $K_k$, which represents the degree of influence of the observed values on the state update:

$$K_k=P_k{H}_k^T{(H{P}_k{H}^T+R)}^{-1}$$

(6)

Finally, update the state vector based on innovation $y_k$ and Kalman gain:

$${\widehat{x}}_k=x_k+K_k{y}_k$$

(7)

For reinforcement learning modeling, the Actor network employs a two-layer GRU to extract historical state sequences within a sliding window $S_t=\left[\begin{array}{ccc}{x}_{t-n},& \dots ,& x_t\end{array}\right](n=10)$. An attention mechanism is applied to assign weights to key frame features: ${\alpha }_t=softmax(W_{2}tanh(W_1·GRU\left(S_t\right)))$, $c_t=\sum {\alpha }_t·GRU\left(S_t\right)$. The policy network generates covariance adjustment parameters based on the contextual feature $c_t$. It outputs the mean $\mu$ and standard deviation $\sigma$ through fully connected layers to define a Gaussian distribution $a_tÑ(\mu ,{\sigma }^2)$ , from which an action $a_t$ is sampled. This action is then used to update the state covariance matrix: $P_{k|k}=P_{k|k}·abs\left(a_t\right)$ . The Critic also employs a GRU combined with an attention mechanism to extract temporal features, evaluate the state value V(s), and uses the position error $r=-\left|\right|{\widehat{x}}_k-x_k^{true}\left|\right|$ as the reward.

The policy update process optimizes the network parameters using the advantage function, where $\gamma =0.99$ is the discount factor:

$$A_t=r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right)$$

(8)

The parameters of the Actor net are updated using the following gradient:

$${\nabla }_{\theta }J\left(\theta \right)=E_t[{\nabla }_{\theta }log\pi \left(a_t\right|s_t;\theta )·A_t]$$

(9)

The parameters of the Critic net are updated by minimizing the mean squared error (MSE):

$$L_{critic}={(r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right))}^2$$

(10)

The overall network architecture is illustrated in Figure 2 below.

3.2. Multi-Trajectory Information Fusion

At the multi-platform level, to achieve cooperative correction of trajectory information among platforms, this paper proposes a fusion strategy based on the information matrix.

First, for trajectory i, its state covariance matrix is $P_i$ and the corresponding information matrix is defined as ${\Lambda }_i=P_i^{-1}$ (in practice, a pseudo-inverse is used to replace the inverse for singular matrices). For the entire UAV swarm, the fused information matrix and state estimate are given by:

$${\Lambda }_{fused}=\sum {\Lambda }_i$$

(11)

$$P_{fused}={\Lambda }_{fused}^{-1}$$

(12)

Considering the differences in covariance among trajectories, the following method is used to compute the mean in order to avoid distortion:

$$P_{mean}={\displaystyle \frac{1}{N}}\sum _{i=1}^n{P}_i$$

(13)

Using the norm $\left|\right|P_{mean}\left|\right|$ as a reference threshold, an asymmetric correction mechanism is adopted. For each trajectory i, if its covariance norm $\left|\right|P_i\left|\right|>\left|\right|P_{mean}\left|\right|$, the trajectory is considered to have high uncertainty. In this case, the fused result is fed back to the original trajectory and updated by weighted averaging:

$$P_i={\displaystyle \frac{1}{2}}(P_{mean}+P_i),\left|\right|P_i\left|\right|>\left|\right|P_{mean}\left|\right|$$

(14)

This enables trajectory estimates with high uncertainty to have their covariance compressed using the fused result, thereby improving overall estimation consistency and system robustness while avoiding the degradation of high-quality estimates.

4. Experiments and Results

4.1. Experimental Setup

The experiments use the IMU data and GNSS signals generated by the simulation software GNSS-INS-SIM (Version 2.1), which produces IMU data and GPS signals. The dataset is divided into training and testing sets. This software can simulate various motion scenarios including static, uniform motion, and accelerated turns. The training dataset contains 20 trajectories covering multiple dynamic motion patterns; the testing dataset contains 5 independent trajectories not included in training, used to evaluate the algorithm’s generalization capability. Additionally, a set of real data collected by the WTGAHRS3-232 (sourced from WitMotion Shenzhen Co, Ltd, Shenzhen, China.) inertial measurement module is introduced for validation. During algorithm training, the Actor-Critic network uses a batch size of 20, learning rates of 1e-5 for the Actor and 1e-4 for the Critic, and a temporal window length sequence-length=10. The Adam optimizer is used for 20 training epochs. The evaluation metric is the three-dimensional root mean square error (RMSE) of position. The effectiveness of the proposed algorithm is validated by comparing its results with those of the EKF and other methods. The Figure 3 below shows the 20 training trajectories and 5 testing trajectories. Trajectories 1–4 are simulated data, while trajectory 5 is real data. Red scatter points represent GPS observations, the green curve denotes error-free ground truth, and the blue curve indicates the estimated trajectory.

4.2. Experimental Methods

The experiment was conducted through the following steps for model training and testing analysis. During the training phase, simulated data was used to construct state sequences (13 dimensions including position, velocity, quaternion, and acceleration). The Actor-Critic network optimized the dynamic adjustment parameters of the Kalman filter covariance matrix, using the negative position error as the reward function. Training proceeded iteratively until the loss function converged. During testing, the trained model was applied to both simulated and real data to calculate the error between the filtered trajectory positions and the ground truth, and to compare the performance of the proposed method against other filtering methods. The Figure 4 (a) below shows the actor loss trend during training, which exhibits three distinct stages: initially, due to random parameters and exploratory strategies, the loss fluctuated sharply; subsequently, as the network learned effective strategies, the loss rapidly increased above zero and the parameter adjustment direction became clearer; finally, the loss stabilized around zero with minor fluctuations, indicating ongoing fine-tuning. The Figure 4(b) critic loss decreased rapidly at first due to large value estimation errors, then fluctuated under data and scenario disturbances, and eventually converged to a low level with small oscillations, reflecting progressively accurate value fitting.

4.3. Experimental Results Evaluation

To evaluate the performance of different filtering methods, this experiment compared the standard EKF, Adaptive Noise-adjusted Kalman Filter (ANKF) [29], Bidirectional Extended Kalman Filter (BEKF) [30], and the proposed RL-IMKF, where the number of trajectories M=5 was used for information matrix fusion. The primary comparison metric was the average position error across each trajectory. Table 1 lists the average position errors for the different filtering methods. The RL-IMKF method demonstrated the lowest error, achieving a 17.4% reduction compared to EKF. This validates that RL-IMKF improves system stability by dynamically generating covariance scaling factors through the Actor-Critic network. Figure 5 illustrates the positional error trends for the same trajectory under different algorithms. The RL-IMKF shows a smoother overall error curve, with both peak values and fluctuation amplitudes smaller than those of EKF, ANKF, and BEKF, demonstrating a clear advantage in stability.

The experimental results indicate that the standard EKF, due to its use of fixed filtering parameters, is prone to model mismatch in dynamic motion scenarios, resulting in the largest errors. Although ANKF introduces an adaptive noise adjustment mechanism, it relies on predefined noise variation models, limiting its adaptability to complex motion patterns. BEKF improves trajectory smoothness through bidirectional filtering but does not consider the credibility differences among multiple trajectories. In contrast, RL-IMKF dynamically optimizes the state covariance matrix of the Kalman filter through reinforcement learning and combines information matrix fusion to leverage multi-trajectory redundancy. This approach not only enables real-time filter parameter optimization but also suppresses error propagation from abnormal trajectories via the cumulative information matrix mechanism, demonstrating superior performance in both training data fitting and test data generalization.

Additionally, to investigate the impact of the number of trajectories on positioning accuracy in multi-trajectory fusion, this experiment selecting trajectory 1 compared the single-trajectory position errors of IMKF with trajectory counts M ranging from 1 to 5. The results are shown in Table 2.

It can be seen that when only a small number of trajectories are fused, the improvement in positioning accuracy is not significant, and error fluctuations may occur due to information redundancy; however, as the number of trajectories increases, the complementary information among multiple trajectories gradually dominates, resulting in a significant decrease in positioning error with the increase in the number of trajectories.

5. Conclusions

This study proposes a Kalman filter localization calibration method based on RL-IMKF, effectively addressing the robustness issue of filter model parameters in dynamic positioning of unmanned clusters. The constructed Actor-Critic network captures temporal features using GRU and attention mechanisms to dynamically generate covariance adjustment factors. Simultaneously, a multi-trajectory information matrix fusion algorithm is designed to aggregate the inverse of trajectory covariance matrices, quantify uncertainty, suppress error propagation, and improve the utilization of multi-source data. Experimental results demonstrate that RL-IMKF achieves the lowest average position error on both training and testing datasets, significantly reducing errors by 17.4% compared to traditional methods such as EKF. Regarding error variation trends, the RL-IMKF error curve is smoother, with peak values and fluctuation amplitudes noticeably smaller than those of other comparative algorithms. Furthermore, a dedicated experiment on the impact of the number of fused trajectories on positioning accuracy shows that reasonably increasing the number of fused trajectories leverages complementary information to reduce errors, providing valuable insights for practical engineering applications.

In practical applications, the RL-IMKF method demonstrates significant potential for adaptation to diverse scenarios. Whether deployed in intelligent unmanned system clusters with stringent environmental perception requirements or in complex environments affected by signal interference and data noise, this method achieves high-precision localization calibration through dynamic optimization of filter parameters and fusion of multi-source trajectory information. From transportation and logistics to monitoring and reconnaissance, RL-IMKF can enhance the reliability and cooperation of unmanned device positioning, facilitating broader and more effective deployment of unmanned systems across multiple fields.

In summary, the RL-IMKF method offers a new approach for high-precision positioning of unmanned clusters. Its strong performance in both theoretical validation and experimental testing provides technical support for the widespread application of intelligent unmanned systems in military, civilian, and other fields. Although this study has achieved significant progress, there remains room for optimization and further development. Future work could explore integrating multi-source heterogeneous sensor data such as UWB, vision, and LiDAR into the algorithm framework, and improve the design of the reinforcement learning state space and information matrix fusion strategies to enable multimodal data collaboration in more complex environments. Additionally, considering the dynamic changes in communication topology during unmanned cluster movement, researching algorithms based on graph neural networks for trajectory association and dynamic fusion will also be a crucial direction to enhance algorithm adaptability.