Adaptive Public Transit Scheduling via Reinforcement Learning: A Survey of Techniques, Challenges, and Opportunities

1. Introduction

The rapid urbanization of cities worldwide has intensified pressure on public transit systems to deliver reliable, efficient, and sustainable mobility. Conventional bus operations depend on pre-computed schedules derived through static optimization–approaches that perform well under predictable conditions but are fundamentally ill-equipped for the stochastic realities of urban traffic characterized by unpredictable passenger arrivals, traffic incidents, and cascading service irregularities such as bus bunching [1,2,3,4,5,6]. The consequence is a well-documented degradation in service quality that drives passengers back to private vehicles, undermining the sustainability objectives that motivate public transit investment in the first place.

Reinforcement learning (RL) offers a better alternative paradigm. Rather than solving a static optimization problem offline, an RL agent learns a control policy through repeated interaction with a dynamic environment, improving its decisions based on delayed feedback. This makes RL inherently suited to sequential decision problems characterized by uncertainty and non-stationarity—precisely the properties that define public bus transit operations.

Scope of this survey. This paper surveys RL applications across five operationally distinct sub-problems in public bus transit, which we define as follows and summarize in Table 3:

1.

Timetable optimization (TTO):The offline or dynamic determination of departure times for each trip on a line, with the objective of minimizing the number of trips (vehicle use) while satisfying passenger demand and service frequency targets.

2.

Vehicle scheduling (VS):The assignment of specific physical vehicles to pre-specified trips, minimizing fleet size, deadhead distance, or operational cost subject to vehicle availability and route compatibility constraints.

3.

Real-time holding and headway control (HHC):Online decisions to hold buses at stops for a specified dwell extension, or to adjust cruising speeds between stops, in order to prevent bus bunching and maintain target headways.

4.

Stop-skipping and dispatching (SSD):Real-time decisions to skip low-demand stops or to adjust the timing of bus departures from terminals to respond to fluctuating demand.

5.

Electric bus charging scheduling (EBCS):The assignment of charging windows to electric buses subject to battery state-of-charge, route schedule, and grid pricing constraints.

These problems differ substantially in their decision variables, time horizons, and optimization objectives. Treating them as a single undifferentiated “scheduling” problem—as is common in prior work—conflates design choices that are problem-specific. This survey maintains these distinctions throughout the literature review.

Why a new survey is needed. RL applications to public transit have been reviewed at varying degrees of breadth in prior work. Haydari and Yilmaz [7] provided an early survey of deep RL across intelligent transportation systems but without focus on bus transit scheduling; their coverage predates the significant methodological growth seen after 2021. More recent surveys on transit optimization [8] focus primarily on classical optimization rather than learning-based approaches. No existing survey systematically maps RL paradigms—value-based, policy-gradient, hierarchical, and multi-agent—to the specific operational problems defined above, using the post-2021 literature. Table 1 summarizes this differentiation. This paper fills that gap.

The contributions of this survey are threefold:

• A precise taxonomy of five operationally distinct bus transit scheduling problems, each with defined decision variables, objectives, and constraints
• A problem-oriented literature review that organizes RL methods according to the transit sub-problem they address, rather than only by algorithm family
• An analysis of open research challenges including sim-to-real transfer, baseline quality, multi-objective reward design, and explainability, with concrete future research directions

The remainder of this paper is organized as follows. Section 2 outlines the systematic review methodology. Section 3 introduces the RL framework and formally defines each transit sub-problem as a Markov Decision Process. Section 4 describes the taxonomy of RL approaches. Section 5 surveys the state-of-the-art organized by transit problem category. Section 6 analyzes cross-cutting findings. Section 7 discusses open challenges and research directions. Section 8 concludes the paper.

2. Materials and Methods

Following the guiding principles of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework [9,10], this section outlines the methodological approach used for the survey, detailing the search strategy, selection of keywords, academic databases consulted, and the specific inclusion and exclusion criteria applied.

2.1. Keywords, Search Strings, and Databases Consulted

To ensure comprehensive coverage of research on reinforcement learning (RL) in public transit scheduling, timetabling, and bunching prevention, a systematic literature search was conducted across major academic databases, including Scopus, Web of Science, IEEE Xplore, and Google Scholar, covering the period from 2021 to October 2025. These databases were chosen to capture a balanced mix of interdisciplinary research, high-impact journal publications, technical conference papers, and high-impact preprints and workshop materials.

Boolean operators were used to combine RL-related and application-specific keywords. The RL-related terms included reinforcement learning, deep RL, multi-agent RL (MARL), actor-critic, Q-learning, policy gradient, PPO, DQN, policy-based RL, and value-based RL. The application-related terms included bus scheduling, timetabling, bus bunching, headway control, dispatching, stop-skipping, holding policy, timetable rescheduling, and fleet scheduling. The general query syntax, adapted to the requirements of each database, followed this structure:

(“reinforcement learning” OR “deep reinforcement learning” OR “RL” OR “MARL”) AND (“public bus” OR “bus scheduling” OR “timetabling” OR “headway control” OR “bus bunching”) AND (year ≥ 2021)

2.2. Screening Procedure

Using the search terms and Boolean expressions described above, the four databases were queried to retrieve a total of 326 articles. After removing duplicates and applying the inclusion and exclusion criteria shown in Table 2, 35 articles were selected for detailed analysis. To complement this process, a word cloud presented in Figure 2 was generated from the keywords extracted from the reviewed studies to highlight dominant themes and emerging research directions in the scheduling of RL-based public transit. The pictorial representation of the overall methodology adopted in this survey is shown in Figure 1

Figure 1. Methodology Diagram

Figure 1. Methodology Diagram

Figure 2. Word cloud summarizing dominant keywords across reviewed studies, showing the prevalence of reinforcement learning, scheduling, optimization,public transit,passenger,and transportation

Figure 2. Word cloud summarizing dominant keywords across reviewed studies, showing the prevalence of reinforcement learning, scheduling, optimization,public transit,passenger,and transportation

Following inclusion screening, each retained study was independently classified according to the transit sub-problem it primarily addresses, drawing on the operational taxonomy introduced in Section 3.1. Studies addressing more than one sub-problem were assigned to the category corresponding to their primary optimization objective and decision variable. This secondary classification step underpins the problem-oriented review structure adopted in Section 5 and ensures that methodological choices are interpreted in context. Of the 35 included studies: 7 address timetable optimization (TTO), 6 address vehicle and fleet scheduling (VS), 14 address real-time holding and headway control (HHC), 5 address stop-skipping and dispatching (SSD), and 3 address electric bus charging scheduling (EBCS).

2.3. Methodological Limitations

This review acknowledges four key methodological limitations. First, a language bias may be present, as only studies published in English were considered, potentially excluding valuable insights from non-English research. Second, the time window spanning 2021 to October 2025, selected to ensure that the survey reflects recent advancements, may have led to the omission of earlier yet significant contributions. Third, the choice of databases could limit coverage, as relevant studies available on alternative or less-indexed platforms might not have been captured. Lastly, the focus on RL as the central methodological framework may have inadvertently excluded important works employing other data-driven or optimization-based approaches to transit scheduling.

3. Reinforcement Learning for Transit Scheduling

In this section, we provide the foundational concepts of RL that underlie the methodologies adopted in the reviewed literature. We first define a precise operational taxonomy of the five transit sub-problems this survey addresses, then formalize the general MDP framework, and finally synthesize problem-specific MDP instantiations that highlight the key structural differences across problem types.

3.1. Operational Taxonomy of Bus Transit Scheduling Problems

The term “scheduling” in the bus transit context encompasses several operationally distinct problems that differ in their decision variables, optimization objectives, time horizons, and feasibility constraints. Conflating these under a single formulation obscures important methodological choices and makes it difficult to assess the appropriateness of a given RL approach. We define five canonical problem categories below; Table 3 summarizes their key characteristics.

Timetable Optimization (TTO). Timetable optimization determines the departure times for each trip across a planning horizon (typically a day or week). The primary decision variable is the set of headways between consecutive departures, and the objective typically involves minimizing the number of scheduled trips (fleet use) while meeting passenger service-level requirements. This is inherently a combinatorial offline planning problem, though recent work has extended it to dynamic timetabling in response to real-time demand [11,12].

Vehicle Scheduling (VS). Given a fixed timetable, vehicle scheduling assigns physical buses to the planned trips. The classical single-depot vehicle scheduling problem (VSP) minimizes fleet size; multi-depot variants additionally minimize deadhead cost [3]. RL-based approaches treat vehicle assignment as a sequential decision process, where the agent selects which bus to dispatch at each departure time [13].

Real-time Holding and Headway Control (HHC). Bus holding is an online, event-driven control strategy in which a bus at a designated control stop is instructed to wait for an additional dwell time before departing. The objective is to maintain target headways between consecutive buses, thereby preventing bus bunching. Unlike TTO and VS, this problem requires real-time decisions with a short time horizon (seconds to minutes) under stochastic travel times and passenger arrival rates. The action space is continuous (holding duration) or discretized [14,15].

Stop-Skipping and Dispatching (SSD). Stop-skipping allows a bus to bypass one or more stops to recover schedule adherence. Dispatching decisions (adjusting departure times from terminals) belong to the same operational category. These strategies operate on a per-trip or per-stop basis under time pressure and are commonly combined with holding in integrated control frameworks [16].

Electric Bus Charging Scheduling (EBCS). The proliferation of battery electric buses introduces a coupled optimization problem: charging windows must be scheduled subject to battery state-of-charge dynamics, service schedule constraints, grid pricing, and charging infrastructure availability. This problem spans multiple time scales, making hierarchical RL formulations particularly relevant [17].

Table 3. Taxonomy of bus transit scheduling problems addressed in this survey.

Problem	Code	Time Horizon	Primary Objective	Key Constraint
Timetable Optimization	TTO	Offline/dynamic	Min. trips / load balance	Headway bounds, capacity
Vehicle Scheduling	VS	Offline	Min. fleet / deadhead cost	Vehicle availability, depot
Holding & Headway Control	HHC	Real-time	Min. headway deviation	Max. holding time
Stop-Skipping & Dispatching	SSD	Real-time	Min. delay + waiting	Service coverage
EB Charging Scheduling	EBCS	Multi-timescale	Min. energy + ops cost	SoC limits, grid capacity

Table 3. Taxonomy of bus transit scheduling problems addressed in this survey.

Problem	Code	Time Horizon	Primary Objective	Key Constraint
Timetable Optimization	TTO	Offline/dynamic	Min. trips / load balance	Headway bounds, capacity
Vehicle Scheduling	VS	Offline	Min. fleet / deadhead cost	Vehicle availability, depot
Holding & Headway Control	HHC	Real-time	Min. headway deviation	Max. holding time
Stop-Skipping & Dispatching	SSD	Real-time	Min. delay + waiting	Service coverage
EB Charging Scheduling	EBCS	Multi-timescale	Min. energy + ops cost	SoC limits, grid capacity

3.2. Reinforcement Learning Framework

Reinforcement Learning (RL) is a machine learning paradigm in which an agent learns to make decisions by interacting with an environment. At each discrete time step t, the agent observes a state $s_t$, selects an action $a_t$, and receives a reward $r_t$ as feedback. Over time, the agent aims to maximize the cumulative reward by learning from trial and error [18].

A standard single-agent RL problem is modeled as a Markov Decision Process (MDP), defined by the tuple $(S,A,P,R,\gamma )$, where:

• S: a finite or continuous set of states. At each time step t, the agent observes $s_t\in S$ representing the current environment condition.
• A: a finite or continuous set of actions. The agent selects $a_t\in A$ to interact with the environment.
• P: a state transition probability function $P\left(s_{t+1}\right|s_t,a_t)$ defining the probability of moving to state $s_{t+1}$ given the current state and action.
• R: a reward function $R(s_t,a_t):S\times A\to \mathbb{R}$ providing immediate feedback.
• $\gamma \in [0,1]$: a discount factor determining the importance of future rewards.

The cumulative return is $G_t:={\sum }_{i=0}^{\infty }{\gamma }^i{r}_{t+i}$. A policy $\pi :S\to A$ guides the agent’s behavior; the objective is to find the optimal policy:

$${\pi }^{*}=arg\underset{\pi }{max}\mathbb{E}\left[{\displaystyle \sum _{t=0}^{\infty }}{\gamma }^t{r}_t\right].$$

(1)

It is very important to note that the MDP tuple above provides a general computational framework, but it does not define a universal model for all five problem categories in Section 3.1. Each problem induces a distinct instantiation: the state space for TTO must encode cumulative passenger demand across a planning horizon, whereas HHC requires only local, real-time vehicle spacing information. The action space in VS is combinatorial (bus-to-trip matching), while in HHC it is a scalar dwell extension. Reward structures differ accordingly: offline scheduling rewards are sparse (end-of-episode cost), while real-time control rewards are dense (per-stop feedback). The generic formulation presented in this section is therefore a conceptual template that must be specialized for each problem, as detailed in Section 3.4.

3.3. Public Transit Scheduling as a Reinforcement Learning Problem

In this context, a transit scheduler acts as an agent interacting with the transit system (the environment) at discrete time steps. The primary objective is to maximize the cumulative reward by learning optimal scheduling decisions. The key elements of this general formulation are:

1. State Space S:

$$s_t=(d_t,v_t,p_t,c_t)$$

(2)

where $d_t$ is passenger demand at different stops/stations; $v_t$ is vehicle locations and statuses; $p_t$ is the planned schedule or headways; and $c_t$ is current traffic conditions.

2. Action Space A:

$$a_t=(\Delta t_{\mathrm{departures}},\Delta r_{\mathrm{reroute}},\Delta h_{\mathrm{headways}})$$

(3)

where $\Delta t$ adjusts vehicle departure times, $\Delta r$ reroutes vehicles to less congested routes, and $\Delta h$ adjusts planned headways.

3. Reward Function $R(s_t,a_t)$:

$$r_t=-\left(\alpha ·\overline{W}+\beta ·C_{\mathrm{ops}}+\gamma ·D_{\mathrm{pen}}\right)$$

(4)

where $\overline{W}$ is average passenger wait time, $C_{\mathrm{ops}}$ is operational cost, $D_{\mathrm{pen}}$ is delay penalty, and $\alpha$, $\beta$ are weighting parameters.

4. Transition Dynamics:

$$s_{t+1}\sim P(·\mid s_t,a_t)$$

(5)

where P represents the (unknown or learned) dynamics of the environment, including passenger behavior and traffic evolution. Table 4 summarizes the key RL components for public transit scheduling.

3.4. Problem-Specific MDP Instantiations

To ground the literature review that follows, we synthesize the key MDP design choices specific to each problem category, identifying the structural choices that distinguish one problem’s RL formulation from another. Table 5 summarizes these distinctions.

MDP for Timetable Optimization (TTO). In TTO, the agent acts as an offline or periodic planner. The state encodes cumulative observed passenger demand at each stop over the planning window, current vehicle capacity utilization, and the residual planning horizon. Actions correspond to headway selection or a binary dispatch decision (dispatch now or wait) for the next scheduled trip. The reward signal is sparse—commonly a terminal reward based on the number of trips used, stranded passenger count, or a composite efficiency score. Credit assignment over long horizons has motivated DQN variants with carefully designed intermediate rewards [11,12].

MDP for Vehicle Scheduling (VS). The agent in VS acts as a dispatcher at each decision point. The state encodes each vehicle’s availability status (remaining working time, rest time, vehicle type, current location), and the action selects which vehicle to assign to the current departure slot. Invalid action masking—pruning vehicles that violate feasibility constraints—is critical to prevent infeasible schedules [13]. Rewards combine a final reward (total vehicles used) with step-wise feasibility penalties.

MDP for Holding and Headway Control (HHC). The agent in HHC operates per-bus at each control point with a short time horizon. The state includes the headway to the preceding bus, the headway to the following bus, on-board passenger count, and stop-level accumulated waiting passengers. The action is a holding duration (continuous or discretized). Rewards are dense, penalizing headway coefficient of variation or average waiting time per step. Multi-agent formulations treat each bus as a separate agent [14,15,19,20].

MDP for Stop-Skipping and Dispatching (SSD). The state encodes current vehicle position, schedule deviation, downstream stop occupancy estimates, and remaining stops on the route. The action is binary (skip stop i, or serve it) or a departure adjustment. Reward design is particularly challenging because skipping reduces operational cost but imposes inequitable access burdens on passengers at skipped stops; recent work has begun to incorporate fairness penalties [16].

MDP for EB Charging Scheduling (EBCS). Hierarchical MDP formulations are most common here. The high-level MDP determines fleet-level charging allocation across a shift (state: battery levels, electricity prices, schedule requirements), while the low-level MDP controls individual charging sessions. Rewards balance energy cost, battery degradation, and schedule feasibility [17].

4. Taxonomy of RL Approaches in Transit Scheduling

This section introduces the taxonomy of RL approaches applied to transit scheduling. We organize approaches along two dimensions: (1) the algorithmic family (value-based, policy-gradient, hierarchical, and multi-agent), and (2) the transit problem category (TTO, VS, HHC, SSD, EBCS) defined in Section 3.1. Table 6 summarizes this two-dimensional mapping and provides the organizing structure for the literature review in Section 5.

4.1. Value-Based RL

In traditional value-based reinforcement learning methods, such as Q-learning, the optimal action-value function $Q^{*}(s,a)$ is approximated through iterative updates on a discrete state-action table. However, in public transit systems the state space is continuous and high-dimensional, encompassing dynamic features such as vehicle trajectories, passenger accumulations, and traffic variations. To handle such complexity, deep reinforcement learning (DRL) introduces a function approximator—typically a deep neural network—to estimate $Q(s,a;\theta )$, where $\theta$ denotes the network parameters. The goal is to minimize the temporal-difference (TD) error:

$$\mathcal{L}\left(\theta \right)={\mathbb{E}}_{(s_t,a_t,r_t,s_{t+1})\sim \mathcal{D}}\left[{\left(y_t-Q(s_t,a_t;\theta )\right)}^2\right],$$

(6)

where the target value is computed as:

$$y_t=r_t+\gamma \underset{a^{\prime }}{max}Q(s_{t+1},a^{\prime };{\theta }^{-}),$$

(7)

and ${\theta }^{-}$ represents the parameters of a slowly updated target network. The replay buffer $\mathcal{D}$ stores past transitions to break temporal correlations, stabilizing learning. This approach, known as the Deep Q-Network (DQN) algorithm [21,22], allows the agent to generalize across vast and continuous transit states without explicit enumeration.

Value-based methods are most naturally suited to discrete-action problems—TTO and VS—where actions such as “dispatch bus now” or “assign vehicle k to trip j” can be enumerated and Q-valued. Their application to continuous HHC requires discretizing the action space, which introduces quantization error. Extensions such as Double DQN, Dueling DQN, and Ape-X DQN address instability and scalability limitations of vanilla DQN in multi-line or multi-depot settings [23,24].

4.2. Policy-Based RL

In policy-based reinforcement learning, the goal is to learn a parameterized stochastic policy ${\pi }_{\theta }\left(a\right|s)$ that directly maps states to probability distributions on actions [25]. The objective is to maximize the expected discounted return:

$$J\left(\theta \right)={\mathbb{E}}_{\tau \sim {\pi }_{\theta }}\left[{\displaystyle \sum _{t=0}^{\infty }}{\gamma }^{t}r(s_t,a_t)\right],$$

(8)

with gradient given by the policy-gradient theorem:

$${\nabla }_{\theta }J\left(\theta \right)={\mathbb{E}}_{{\pi }_{\theta }}\left[{\displaystyle \sum _{t=0}^{\infty }}{\nabla }_{\theta }log{\pi }_{\theta }\left(a_t\right|s_t)G_t\right].$$

(9)

A baseline $b\left(s_t\right)$, often the value function $V_{\varphi }\left(s_t\right)$, can be subtracted to reduce variance:

$${\nabla }_{\theta }J\left(\theta \right)\approx \mathbb{E}\left[{\nabla }_{\theta }log{\pi }_{\theta }\left(a_t\right|s_t)\left(G_t-b\left(s_t\right)\right)\right].$$

(10)

In public transit scheduling, where many actions (e.g., dwell time or dispatch delay) are continuous, a deterministic policy ${\mu }_{\theta }\left(s\right)$ can be adopted, with gradient:

$${\nabla }_{\theta }J\left(\theta \right)={\mathbb{E}}_{s\sim d^{\mu }}\left[{\nabla }_{\theta }{\mu }_{\theta }\left(s\right){\nabla }_a{Q}^{\mu }(s,a){|}_{a={\mu }_{\theta }\left(s\right)}\right].$$

(11)

Policy-gradient methods—particularly PPO, SAC, and DDPG—are better matched to HHC and EBCS problems, where actions (holding duration, charging power level) are inherently continuous. The entropy bonus in SAC promotes persistent exploration, which is valuable in non-stationary environments where passenger demand shifts across time-of-day or day-of-week [12,26]. Actor-critic formulations with centralized training and decentralized execution (CTDE) have become the dominant paradigm for multi-bus coordination, preserving scalability at execution time [19,20].

4.3. Hierarchical RL

Hierarchical reinforcement learning (HRL) decomposes a complex decision problem into a hierarchy of sub-problems operating at different temporal scales. A high-level policy selects options—temporally extended sub-goals—while a low-level policy executes primitive actions to achieve them. This structure is particularly well-suited to EBCS, where fleet-level charging allocation (hours) and individual session control (minutes) operate at fundamentally different time scales, and to integrated timetabling and dispatching problems where strategic and tactical decisions must be coordinated.

In the surveyed literature, DAC-MAPPO-E [17] exemplifies this approach by reformulating EB charging scheduling as a two-level MDP: the high-level agent allocates charging windows across the shift, and the low-level agents control per-session charging power. Similarly, HMARL [27] uses a two-layer policy—a high-level discrete strategy selector and a low-level continuous controller—for bus bunching control. The key advantage of HRL is its ability to reduce the effective planning horizon at each level, alleviating the credit-assignment problem over long episodes. The key limitation is the added complexity of specifying sub-goal structure and ensuring stable co-training between levels.

4.4. Multi-Agent RL (MARL)

Public transit systems are inherently multi-agent: multiple buses operating on shared or interconnected routes must coordinate their holding, dispatching, or charging decisions. MARL formalizes this as a decentralized partially observable Markov decision process (Dec-POMDP), where each agent (bus) observes a local state and selects actions to maximize a shared or individual reward.

The dominant MARL paradigm in the surveyed literature is CTDE, implemented via actor-critic frameworks such as MA-DDPG, MA-PPO, and CAAC [14,19,20]. CTDE allows agents to share global state information during training, facilitating coordination, while executing independently during deployment—an important practical constraint given onboard computation limits. A critical challenge is non-stationarity: from any single agent’s perspective, the environment is non-stationary because other agents are simultaneously learning, violating the stationarity assumption underlying standard RL convergence guarantees. Recent work has addressed this through parameter sharing [28], graph attention critics [19], and single-agent reformulations that encode inter-agent context in the state representation [12].

4.5. Explainability and Institutional Considerations

The deployment of RL-based scheduling systems in operational contexts requires more than technical performance: transit agencies must be able to explain, audit, and justify algorithmic decisions to regulators, passengers, and policymakers. This creates a distinct demand for explainable reinforcement learning (XRL)—methods that render RL policies interpretable without sacrificing performance.

In the bus transit domain, the first dedicated XRL approach for timetable optimization was introduced by Ai et al. [29], who augmented a DQN-based timetabling agent with post-hoc explanation mechanisms that identify which state features (e.g., current load factor, time of day, stop-level demand) most strongly influenced each dispatch decision. Their XRL-TO framework demonstrated that interpretability and optimization quality need not be traded off: the explained policy retained 97 % of the unexplained agent’s performance on real Xiamen bus line data while producing human-readable decision rationales. More broadly, XRL approaches can be categorized as intrinsic (interpretable-by-design policies, such as decision-tree surrogates) or post-hoc (attribution methods such as SHAP applied after training) [30]. Both directions are relevant for transit scheduling: intrinsic methods align better with operational requirements for transparency, while post-hoc methods are more readily applicable to existing deep RL pipelines.

5. Literature Survey

Reinforcement learning has been applied across the five problem categories defined in Section 3.1 with markedly different levels of maturity, algorithmic diversity, and empirical rigor. This section reviews the state-of-the-art organized by problem type, enabling readers to track the research trajectory specific to each operational challenge.

5.1. Timetable Optimization (TTO)

Timetable optimization is the most algorithmically mature application of RL to bus transit, with a clear progression from single-agent DQN formulations toward sophisticated architectures incorporating future-aware mechanisms and explainability.

Early DQN approaches. The foundational RL-based timetabling work by Ai et al. [11] frames bus timetable optimization as a per-minute dispatch decision problem, where the agent chooses whether to send a new bus service at each time step. Operating on real AFC data from Xiamen City Lines 2, 230, and 239, the DQN agent with a carefully designed composite reward (incorporating load factor, empty load rate, waiting time, and stranded passengers) achieved an 8 % reduction in vehicle use and 17 % reduction in average waiting time compared to the operator’s existing schedule. A key design choice was constraining the RL step interval to match real operational decision frequencies—a practical consideration often overlooked in purely academic implementations.

Distributed multi-agent timetabling. Yan et al. [23] extended the DQN framework to multi-line dynamic timetabling using an Ape-X DQN architecture, which enables distributed off-policy learning across parallel simulation workers. Applied to Beijing bus Lines 404 and 593, this approach achieved a 20.3 % reduction in combined operational and passenger costs compared to actual operator schedules. The distributed architecture addresses a known limitation of single-worker DQN: slow experience generation on computationally expensive transit simulators.

Future-aware architectures. Zhang et al. [31] introduced FA-DQN, which augments the standard DQN pipeline with two innovations: Kolmogorov-Arnold Networks (KAN) to improve modeling of nonlinear passenger flow dynamics, and a Mamba-based future-aware mechanism that conditions dispatch decisions on predicted short-term demand. Applied to a Beijing multimodal bus-subway network, FA-DQN achieved the highest cumulative reward among baselines and reduced CO₂ emissions by 68 % (bus) and 27 % (subway). The self-adaptive headway-based action selection,rather than binary dispatch decisions, simplifies learning for longer headway intervals.

Explainability in timetabling. The most recent advance in TTO is XRL-TO [29], which addresses the institutional barrier to deployment by augmenting a DQN-based timetabling agent with post-hoc explanation mechanisms. This work establishes that explainability and performance are not mutually exclusive in the timetabling context, and is the first paper to apply XRL principles specifically to bus timetable optimization.

Baseline quality assessment. A notable limitation across TTO studies is the inconsistency of baselines. Ai et al. [11] compare against the operator’s existing fixed schedule, which represents current practice but not an optimized alternative. Yan et al. [23] compare against historical operational data. FA-DQN [31] includes comparisons with standard DQN but does not benchmark against classical OR methods such as MILP-based timetabling, which are known to find near-optimal solutions for the offline problem [2,5]. Future TTO work should systematically include at least one strong OR baseline to establish whether RL’s gains come from the adaptivity of the learning approach or simply from reward shaping relative to the fixed schedule. Table 7 summarizes the surveyed works on timetable optimization.

5.2. Vehicle and Fleet Scheduling (VS)

Vehicle scheduling, which basically means,assigning physical buses to pre-determined trips, has received less RL attention than timetabling or headway control, but the available work covers both classical and electric bus variants.

Classical single-depot scheduling. The RL-BSA framework of Liu et al. [13] represents the first application of RL specifically to the bus scheduling problem (BSP). Framing each departure time as a sequential decision point, the D3QN agent selects which vehicle to assign at each slot, guided by state features encoding vehicle remaining working time, rest time, type, and executed trip count. An invalid action masking layer eliminates infeasible assignments without requiring explicit penalty shaping. Experiments on real Chinese bus lines demonstrate that RL-BSA matches or exceeds adaptive large neighborhood search (ALNS) heuristics in vehicle minimization while adapting online to disruptions, which is a capability unavailable to static offline schedulers.

Multi-depot electric bus scheduling. Yan et al. [37] address the considerably harder multi-terminal, multi-route electric bus (EB) scheduling problem. Their hybrid RL+MILP framework uses Q-learning or TD3 at the high level to determine the number of deadhead runs, reducing the problem to a simpler sub-problem solvable by MILP. This hierarchical decomposition,RL for strategic decisions, OR for feasibility-constrained tactical assignments,is an important design pattern for problems where constraint satisfaction is non-negotiable. Applied to a Hong Kong bus network, the hybrid approach reduced missed service trips by 36.73 % compared to a rolling horizon optimizer (RHO).

Fleet allocation under demand uncertainty. Li et al. [40] address bus fleet allocation across multiple routes using a distributionally robust optimization (DRO) model combined with a multi-operator genetic algorithm enhanced with RL (MOGA-RL). The RL component learns operator selection weights for the genetic algorithm, accelerating convergence on large-scale problems. Real-world experiments on Beijing’s public bus system show that DRO provides more conservative and reliable allocations than deterministic models under demand uncertainty.

Multi-agent customized bus routing. Wu et al. [39] address the responsive customized bus route planning problem,a variant of VS where routes are planned dynamically in response to real-time passenger requests. Their MRL-RP framework uses multi-agent deep RL where each agent plans one route, enabling parallel, coordinated route construction. This work extends the VS scope beyond fixed-route assignment to demand-responsive operations, an important direction as transit agencies increasingly offer flexible service tiers.

Baseline quality assessment. VS is the sub-field where OR baselines are most consistently used. Liu et al. [13] compare against ALNS; Yan et al. [37] compare against a rolling horizon optimizer. However, comparisons against the classical exact VSP solver (network flow formulation [3]) are absent from all RL-based VS papers,a gap that makes it difficult to quantify how much optimality is sacrificed for the adaptivity RL provides. A summary of the surveyed work on bus and fleet scheduling is presented in Table 8

5.3. Real-Time Holding and Headway Control (HHC)

HHC is the most extensively studied RL application in bus transit, with the largest and most methodologically diverse body of literature. Research has progressed from single-agent tabular approaches to sophisticated multi-agent frameworks with graph-based coordination.

Tabular and early DRL approaches. Comi et al. [42] propose two multi-agent RL algorithms,one based on SARSA, one using an artificial neural network,for combined holding and speed control on a single bus route. Operating on AVL-calibrated simulated data, the approach eliminates bus bunching and improves regularity. He et al. [32] apply a modified Q-learning with multi-stage look-ahead (ADP) for headway stabilization across five simulated bus lines, demonstrating that even classical tabular methods remain competitive when the state space is carefully designed.

Parameter-sharing DQN. Liu et al. [28] formulate bus holding as a decentralized partially observable MDP and develop PS-DQN, which shares network parameters across agents to improve sample efficiency in multi-bus settings. An event-driven simulator accommodates stochastic travel times and demand patterns. PS-DQN achieves the lowest headway variability against occupancy-threshold (OT) and occupancy-headway (OH) baselines. The event-driven simulator design is a practical contribution: it avoids the computational waste of fixed-timestep simulation when bus arrivals at stops are sparse.

Spatial-temporal deep RL. Zhao et al. [43] propose a deep RL approach for bus holding control that explicitly incorporates spatial-temporal data,specifically, historical travel time patterns encoded as spatiotemporal features, into the state representation. This allows the agent to condition holding decisions on predicted future travel conditions, not just current headways. Evaluated on an Australasian bus network, the approach outperforms schedule-adherence and headway-based holding baselines.

Multi-agent actor-critic frameworks. Wang and Sun produced the most sustained research program in MARL for bus bunching, spanning four papers [14,19,20,38]: MPPO-T (joint action tracking for asynchronous coordination), CAAC (graph attention network for credit assignment), CAAC extended to multi-line shared corridors with preference-weighted multi-objective rewards, and a distributional MARL approach (IQNC-M) for robustness under extreme disruptions. A critical finding from this body of work is that simpler state representations can match or outperform complex ones [15], and discrete action spaces are practically preferable to continuous ones despite theoretical disadvantages.

Single-agent reformulation. Zhang [12] challenges the MARL consensus by showing that a single SAC agent with augmented state encoding (vehicle ID, stop ID, time period as categorical embeddings) matches or exceeds MARL baselines while avoiding non-stationarity and data imbalance problems. This is an important methodological insight: multi-agent complexity may be unnecessary when inter-agent dependencies can be encoded in the state.

Integrated multi-strategy control. Yu et al. [27] propose HMARL, which uniquely combines both holding and speed control (via dedicated bus lanes) in a hierarchical two-layer policy. The high-level agent selects a control strategy (hold or accelerate), and the low-level agent selects parameters. An RNN-based duration-critic handles asynchronous, temporally extended actions. This is the most realistic operational model among surveyed HHC papers, as it acknowledges that holding alone is insufficient when buses are already in motion.

Baseline quality assessment. HHC has the strongest baseline comparisons of any sub-field in this survey. Most papers compare against no-holding (NH), schedule-based (SB), headway-based (HB), and equal-headway (EH) policies. Xu et al. [15] provide the most rigorous evaluation, comparing Double DQN, DDPG, MA-DDPG, and MA-PPO under identical simulation conditions with real Chengdu data. The primary gap is the absence of comparisons with model-predictive control (MPC) methods, which represent a strong non-RL baseline for real-time holding control. Table 9 captures the summary of the reviewed literature under this category.

5.4. Stop-Skipping and Dispatching (SSD)

Stop-skipping and dispatching decisions are typically studied alongside holding control, but their distinct decision structure (binary skip/serve vs. continuous dwell extension) warrants separate treatment.

Multi-strategy real-time control. Rodriguez et al. [16] present the most comprehensive joint treatment of holding and stop-skipping. Their DDQN-HA framework operates on the Chicago transit network using real-world data, with a reward function that penalizes waiting time, riding time, and the long-term equity costs of denied boardings at skipped stops. DDQN-HA achieved a 3.2 % average waiting time reduction and 6.5 % reduction in reliability buffer time compared to equal-headway holding. Crucially, the paper acknowledges that stop-skipping’s equity implications—some passengers are systematically excluded from service—require explicit reward treatment, not just efficiency optimization.

Customized bus route planning. Wu et al. [39] extend the SSD scope to demand-responsive customized bus services, where the route itself is partially determined in real time. Their multi-agent DRL framework (MRL-RP) plans routes dynamically in response to passenger requests, treating route selection as a sequential dispatching decision. This represents the frontier of SSD research: moving from fixed-route control to flexible-route design.

Baseline quality assessment. SSD papers generally compare against schedule-based control and simple rule-based skipping heuristics. No surveyed SSD paper benchmarks against mathematical programming formulations of the skip-stop problem, despite the existence of well-established ILP-based skip-stop optimizers in the OR literature. This gap is significant: the RL advantage in SSD may partly stem from comparison against weak baselines rather than algorithmic superiority. Table 10 show the summary of the surveyed papers under stop-skipping and dispatching transit operation.

5.5. Electric Bus Charging Scheduling (EBCS)

EBCS is the newest RL application area in this survey, with research concentrated after 2022 as electric bus adoption accelerated globally.

Hierarchical multi-agent charging. Qi et al. [17] propose DAC-MAPPO-E, a hierarchical DRL framework for electric bus charging scheduling. The high-level policy allocates charging windows across the shift; the low-level policy controls per-session power levels. Attention-based scalability mechanisms allow the framework to handle fleet sizes not seen during training. Applied to real electricity pricing data from MISO and Guelph Transit schedule data, DAC-MAPPO-E achieves within 0.18 % of the theoretical MILP optimum while scaling to large fleets where MILP becomes computationally intractable.

RL+MILP hybrid for multi-terminal EB scheduling. Yan et al. [37], discussed in Section 5.2, are also relevant here: their hybrid approach to multi-terminal EB scheduling incorporates battery constraints, flexible charging opportunities, and stochastic travel times—challenges specific to electric fleets.

Baseline quality assessment. EBCS papers compare primarily against MILP formulations, which is the appropriate gold standard given that exact methods remain tractable at moderate fleet sizes. DAC-MAPPO-E’s comparison against the MILP optimum is the most rigorous evaluation in the survey. A missing dimension is comparison against rule-based charging policies (e.g., charge whenever SoC drops below a threshold), which represent current practice in many transit agencies. The summary of the surveyed literature is presented in Table 11

6. Analysis of Survey Findings

To complement the review in Section 5, this section presents a cross-sectional analysis of the 35 studies in our corpus. The aim is to expose measurable trends, methodological diversity, and research gaps across the RL approaches applied to public-transit scheduling.

6.1. Algorithmic Trends

In total, 35 studies were analyzed, comprising 19 primarily value-based and 16 primarily policy-based implementations, as shown in

Figure 3. Although the difference between the two categories is relatively small, the progression of research over time suggests a gradual shift from classical value-function optimization methods toward policy-gradient and actor-critic frameworks (e.g., PPO, SAC, MAPPO). Between 2021 and 2022, most works employed value-based approaches, coinciding with the early adaptation of deep RL architectures for transit operations. Starting from 2023, the number of policy-based applications began to increase steadily, reaching near-parity by 2025.

The shift toward policy-gradient methods after 2023 is not merely a reflection of algorithmic fashion; it tracks a genuine change in the operational complexity of the problems being tackled. Early RL work in bus transit addressed single-route, single-objective problems with discrete action spaces—settings where DQN’s stability and sample efficiency provided clear advantages. As the field moved toward multi-line coordination, continuous holding decisions, and multi-objective trade-offs, the limitations of discrete value-based methods became apparent. Policy-gradient methods, particularly PPO and SAC, offer smoother optimization landscapes and native support for continuous actions, making them better suited to the operational complexity of realistic transit networks. This evolution mirrors transitions observed in robotics and autonomous systems RL, where continuous control problems systematically favor policy-gradient over value-based approaches.

Hierarchical RL remains underexplored relative to its potential. Only two of the 35 surveyed papers explicitly use hierarchical MDP formulations [17,27], despite the fact that transit operations naturally decompose into strategic (timetabling, fleet allocation) and tactical (holding, dispatching) decision layers. This is a clear research gap.

6.2. Data Utilization and Geographic Distribution

With respect to data utilization, the field exhibits a clear bias toward empirical validation. Twenty-four of the thirty-five studies used real-world datasets, while eleven relied on synthetic simulations as shown in Figure 4 This dominance of real data suggests that RL-based scheduling and headway control methods are increasingly being evaluated under realistic operational conditions, often through integration with automatic vehicle location (AVL) or smart-card datasets. This trend underscores the growing maturity of the field, moving beyond proof-of-concept experiments toward deployable decision-support tools for urban transport agencies.

The geographic concentration of real-world datasets in Chinese cities as shown in Figure 4 is not solely a data availability artifact—it reflects a structural asymmetry in research funding and transit system scale. Beijing, Nanjing, and Chengdu operate high-frequency, high-demand bus networks with centralized data infrastructure (AFC smart-card systems, AVL tracking), which are prerequisites for training data-hungry deep RL models. European and North American systems, while well-instrumented, tend to operate at lower frequencies and are subject to stricter data governance. This creates a genuine transferability challenge: an RL policy trained to maximize headway regularity on a high-frequency Beijing corridor may behave poorly on a lower-frequency European suburban route where the bunching phenomenon itself rarely occurs.

6.3. Baseline Quality Analysis

A systematic assessment of comparison baselines across the 35 surveyed papers reveals a consistent pattern that warrants critical attention. Table 12 classifies the baselines used in each problem category by their strength: (1) weak heuristics (fixed schedules, rule-based policies, no-action baselines), (2) meta-heuristics (genetic algorithms, simulated annealing, ALNS), and (3) state-of-the-art OR methods (MILP, ILP, network flow formulations, MPC).

The overall picture is that RL in bus transit is predominantly compared against weak heuristics representing current practice, not against the best achievable solutions from classical optimization. While this is partially justified, RL’s value proposition is adaptivity in real-time, not offline optimality,it makes it difficult to assess whether RL genuinely advances the state of the art or simply improves on the status quo. Future work should, where computationally feasible, include at least one strong OR baseline: MILP for offline problems, MPC for real-time control, or ALNS for combinatorial scheduling.

7. Open Challenges and Future Research Directions

The literature reviewed in this survey establishes several positive findings: RL agents consistently outperform fixed-schedule and rule-based holding policies on the metrics they are optimized for; policy-gradient and actor-critic methods are increasingly competitive with and superior to value-based methods for continuous-action control; and real-world validation has become the norm rather than the exception. However, the gap between algorithmic demonstration and operational deployment remains wide. The following challenges, synthesized from the limitations identified across the reviewed studies, must be resolved for RL-based transit scheduling to transition from research to practice. Table 13 maps each challenge to a concrete emerging research direction.

7.1. Regional Imbalance and Data Representativeness

An important observation from the survey is that most RL applications for public transit scheduling are validated using real-world datasets rather than synthetic simulations, indicating growing practical maturity. However, a clear geographical concentration emerges: the majority of these datasets originate from Chinese cities such as Beijing, Nanjing, Chengdu, Xiamen, and Hong Kong. Only a small number of studies draw on data from Western contexts, such as Rome or Chicago transit networks.

This regional skew raises questions about the generalizability and transferability of current RL models. Urban transport systems vary greatly in terms of infrastructure, regulatory environments, data collection technologies, and service patterns. A model trained on dense, high-frequency bus networks typical of East Asian megacities may not perform effectively in European or North American systems characterized by lower service frequency, multimodal integration, and stricter privacy regulations.

Future research should focus on diversifying the empirical foundations of RL for public transit. Collaborative projects with European, American, and African transport agencies could expand the diversity of available datasets and operational contexts. Moreover, federated learning and privacy-preserving data-sharing frameworks could allow researchers to train models across regions without direct data exchange, overcoming institutional and legal barriers. A concrete research direction is the development of domain adaptation techniques that allow models trained on data-rich Chinese cities to transfer to data-scarce systems in Africa, South Asia, and Latin America—regions where public transit is both most critical and least studied.

7.2. Sample Inefficiency and Convergence Instability

Many RL algorithms require extensive training interactions to reach acceptable performance levels, resulting in high computational costs and energy consumption. Moreover, instability in convergence is frequent, especially under changing passenger demand or fluctuating traffic conditions, making practical deployment computationally prohibitive. Future work should explore model-based and offline RL methods that reduce sample requirements by leveraging learned dynamics or pre-existing data. Transfer learning and meta-learning could further accelerate convergence by reusing policies across similar transit networks or operational contexts.

A promising but underexplored direction is offline RL—training RL policies entirely from historical operational logs without live environment interaction. Existing AVL and AFC datasets from transit agencies represent millions of bus-stop events and could, in principle, support offline policy learning without costly simulation. Conservative offline RL methods such as CQL or IQL are particularly relevant here, as they guard against distributional shift between the historical dataset and the learned policy.

7.3. Multi-Agent Coordination

Public transport systems are inherently multi-agent in nature, involving numerous buses, depots, and traffic control systems that must coordinate actions efficiently. Existing MARL frameworks face scalability and communication challenges as the number of agents increases. Future research should advance graph-based MARL architectures that explicitly model inter-agent relationships through graph neural networks. This approach can improve coordination efficiency, reduce redundant communication, and improve system stability as networks scale.

The single-agent reformulation of Zhang [12], which encodes inter-agent context in an augmented state rather than using explicit multi-agent training, deserves further investigation as a scalable alternative to MARL for HHC. If this approach generalizes to larger networks, it would substantially reduce the engineering complexity of deploying coordination policies.

7.4. Multi-Objective Optimization

Most RL studies for transit scheduling focus on a single performance indicator, such as minimizing delay or waiting time. In reality, transport management requires balancing competing objectives that include passenger comfort, operational costs, energy use, and emissions. A viable future direction is the adoption of multi-objective RL frameworks capable of producing Pareto-optimal policies. These methods can allow transit planners to navigate trade-offs among multiple goals without reducing them to a single scalar reward.

7.5. Interpretability and Institutional Trust

The opaque nature of RL decisions presents a barrier to acceptance by transport agencies, which must justify their scheduling policies to regulators and the public. The absence of transparent reasoning mechanisms raises concerns about safety, accountability, and fairness. Research into explainable reinforcement learning (XRL) can help bridge this gap by visualizing policy behaviors, identifying key state features influencing decisions, and linking actions to reward attributions [30]. Integrating human expertise through human-in-the-loop frameworks can ensure that algorithmic choices remain consistent with established operational standards and policy objectives.

The XRL-TO framework [29] provides the first evidence that explainability and performance are not mutually exclusive in bus transit scheduling, achieving near-parity with the unexplained agent while producing human-readable decision rationales. Extending XRL principles from timetable optimization to real-time holding and dispatching control—where decisions must be explained to drivers in seconds—is a critical open research direction.

7.6. Sim-to-Real Transfer

The gap between simulation-based RL training and real-world deployment—the sim-to-real gap—is one of the most consequential unsolved problems in applied RL, and its implications for bus transit scheduling are under-discussed in the current literature.

Simulator fidelity. The majority of reviewed papers use custom microsimulators or SUMO-based environments. These simulators necessarily simplify aspects of real transit operations: passenger arrival processes are typically modeled as Poisson or uniform distributions, while real arrivals cluster around timetabled connections and school or work schedule peaks. Traffic conditions are often represented as aggregate congestion levels rather than vehicle-level interactions. The consequence is that RL policies learn to exploit statistical regularities present in simulation but absent or structured differently in reality. Xu et al. [15] mitigate this partially by calibrating their simulator to real Chengdu GPS and smart-card data, representing best practice.

Safety and operational constraints. Real-world bus operations are subject to hard constraints such as maximum holding times, headway bounds imposed by timetable contracts, and crew duty time regulations that are difficult to enforce as soft reward penalties without risking policy convergence to constraint-violating solutions. Constrained RL methods (e.g., Lagrangian primal-dual approaches, safety-layer architectures) have been applied successfully in robotics and autonomous vehicles for constraint satisfaction, but their application to transit scheduling remains unexplored.

Hardware latency. Real-time holding decisions must be communicated to drivers within seconds of a bus arriving at a control stop. Deep RL inference on embedded onboard hardware introduces latency that is absent from simulation. For large MARL systems requiring centralized value estimation (CTDE), communication delays between the central server and individual buses add further uncertainty. Policy distillation, a method in which a trained deep RL policy is compressed into a lighter surrogate model and edge deployment techniques are practical mitigations, but no surveyed paper evaluates inference latency as a deployment metric.

Recommended research directions. Three specific directions are most promising: (1) domain randomization during training, systematically varying simulator parameters (passenger arrival rates, dwell time distributions, sensor noise) to train policies robust to model mismatch; (2) real-world fine-tuning protocols that use a small number of live operational episodes to adapt simulation-trained policies to deployment environments; (3) constrained RL formulations that enforce operational safety requirements as hard constraints rather than reward penalties, preventing constraint violation during both training and deployment.

7.7. Large Language Models and Generative AI in Transit Scheduling

The emergence of large language models (LLMs) as general-purpose reasoning systems opens concrete, non-speculative opportunities for augmenting RL-based transit scheduling. We identify three specific integration pathways supported by recent published work.

LLM-assisted reward engineering. Designing reward functions for complex transit scheduling problems is labor-intensive and requires expert knowledge of operational trade-offs. Recent work demonstrates that LLMs can generate and iteratively refine RL reward functions from natural language problem descriptions. The LLM-enhanced RL framework for bus holding control of Shen et al. [45] shows that LLM-generated reward specifications can outperform manually designed rewards on standard bus bunching benchmarks, reducing human engineering effort while improving policy quality. This is analogous to the EUREKA system [46], which used LLMs to generate reward functions for robotic manipulation tasks—a proof of concept directly applicable to transit scheduling.

LLM-assisted scenario generation. Training robust RL policies requires exposure to diverse and rare operational scenarios like major disruptions, cascading delays, and extreme demand surges that may be underrepresented in historical data. LLMs can generate realistic operational scenario descriptions that can be translated into simulation parameters, enriching the training distribution. This application requires no real-time LLM inference, sidestepping the latency objections that limit LLM use as direct transit controllers.

LLM-as-explainer. Post-hoc LLM-generated natural language explanations of RL decisions represent a practical path toward interpretability that does not require intrinsically interpretable policies. Given a trained RL agent’s state-action trajectory and reward signals, an LLM can generate human-readable summaries of the agent’s decision logic, for example, “the bus was held for 45 seconds because the trailing bus was 2 minutes behind schedule and upstream demand was elevated”—which can be presented to drivers and operators via onboard displays.

7.8. Ethical and Sustainability Considerations

Few studies explicitly incorporate social equity or environmental sustainability into their design objectives. Policies optimized solely for travel efficiency risk deprioritizing low-demand routes or increasing emissions through more aggressive vehicle dispatching. Future research should embed the principles of sustainable mobility directly into the learning process. This includes formulating reward functions that capture fairness, accessibility, and environmental impact, ensuring that RL advances align with broader societal and ecological objectives, similar to the work in [47] but in the public bus scheduling context.

8. Conclusions

This survey has examined the growing body of research applying reinforcement learning (RL) to public transit scheduling, a domain historically dominated by mathematical programming and heuristic optimization. RL introduces a paradigm shift: instead of pre-computing fixed schedules, it learns adaptive control policies capable of responding to real-time demand, disruptions, and network dynamics.

Through systematic review, we identify five key public transit operations where RL is applied namely timetable optimization, vehicle (bus) and fleet scheduling, stop-skipping and dispatching, real-time holding and headway control, and electric bus charging scheduling. Further analysis shows that two dominant RL methodological families were identified: value-based methods (e.g., DQN and its variants) and policy-based or actor-critic methods (e.g., PPO, DDPG, A3C). Each family exhibits distinct strengths: value-based approaches provide stability and sample efficiency, whereas policy-gradient methods offer flexibility for continuous and stochastic action spaces. The analysis revealed that while RL demonstrates potential for improved service reliability and reduced passenger delay, real-world deployment remains rare. Challenges such as limited or regional imbalance in open data, sample inefficiency, interpretability, and hardware constraints continue to hinder translation from simulation to operation. These limitations underscore the need for integrated, interdisciplinary research combining transportation engineering, machine learning, and operations management.

With regard to future work, we intend to address the challenges mentioned in Section 7.