To enhance system reliability and mitigate vulnerabilities of the Global Navigation Satellite Systems (GNSS), it is common to fuse the Inertial Measurement Unit (IMU) and visual sensors with the GNSS receiver in the navigation system design, effectively enabling compensations with absolute position and reducing data gaps. To address shortcomings of traditional Kalman Filter (KF) such as sensor error, imperfect nonlinear system model, and KF estimation error, a GRU-aided ESKF architecture is proposed to enhance positioning performance. This study conducts Failure Mode and Effect Analysis (FMEA) to priories and identify the potential faults in the urban environment facilitating the design of improved fault-tolerant system architecture. The identified primary fault events are data association error and navigation environment error during fault conditions of feature mismatch, especially in the presence of multiple failure modes. A hybrid federated navigation system architecture is employed using a Gated Recurrent Unit (GRU) to predict state increments for updating the state vector in the Error Estate Kalman Filter (ESKF) measurement step. The proposed algorithm’s performance is evaluated in a simulation environment in MATLAB under multiple visually degraded conditions. Comparative results provide evidence that the GRU-aided ESKF outperforms standard ESKF and state-of-the-art solutions like VINS-Mono, End-to-End VIO, and Self-Supervised VIO, exhibiting higher efficiency in complex environments.