A Review of Ship Path Planning for Autonomous Navigation: From Model-Driven Methods to Deep Reinforcement Learning

1. Introduction

Unmanned surface vehicles (USVs) and maritime autonomous surface ships are moving from experimental platforms to operational use in ocean monitoring, patrol, transport, surveying, and emergency response. Their value lies not simply in replacing onboard crews, but in sustaining navigation in hazardous and complex environments. This requires more than point-to-point guidance: an autonomous vessel must continuously balance safety, route efficiency, regulatory compliance, and manoeuvrability under uncertain sea states and dynamic traffic. Reconciling these coupled, and sometimes competing, objectives is a central challenge in autonomous navigation [1,2,3,4,5].

Ship path planning is therefore fundamentally different from path search for ground robots. Navigable waters are constrained by coastlines, shallow regions, reefs, fairways, and port infrastructure, while ship inertia, underactuation, and delayed steering can render a geometrically collision-free path dynamically infeasible. Other vessels are also interactive agents rather than passive obstacles: their behaviour reflects navigational intent, manoeuvring capability, and responsibilities defined by the International Regulations for Preventing Collisions at Sea (COLREGs). Wind, waves, currents, sensing errors, and communication delays further complicate state estimation and motion prediction. Ship path planning is thus best formulated as a constrained sequential decision problem that couples spatial feasibility, collision risk, rule compliance, and vessel dynamics [6,7,8,9,10,11].

Methods have accordingly progressed from static route search to dynamic collision avoidance, continuous manoeuvring, multi-vessel interaction, and system-level safety assurance. Search, sampling, geometric, and optimization methods remain valuable because they provide explicit structure and interpretable constraints, whereas deep reinforcement learning (DRL) can represent high-dimensional states and complex interactions that are difficult to model analytically. These paradigms are increasingly combined rather than treated as alternatives: electronic navigational charts, collision-risk metrics, COLREGs, vessel dynamics, and optimization constraints are embedded into learning systems as route priors, state features, reward terms, action constraints, and safety filters. The emerging direction is therefore hybrid decision-making that couples model-based knowledge, interaction learning, and verifiable safety [12,13,14,15,16,17].

Existing methods can be organized along two complementary dimensions: decision basis and planning scope. Accordingly, they fall into five broad classes: search- and sampling-based, geometry- and rule-based, optimization-based, learning-driven, and hybrid or hierarchical methods. Figure 1 maps these classes across the axes of “global route planning–local manoeuvring” and “explicit models–interaction learning,” while Table 1 summarizes their functions, strengths, and applicability. The categories are complementary rather than mutually exclusive. Search and sampling methods mainly support global route generation; geometric and rule-based methods provide rapid risk assessment and regulatory enforcement; optimization methods handle dynamic and multi-objective constraints explicitly; learning-based methods address high-dimensional states and complex interactions; and hybrid frameworks combine adaptability with stronger safety guarantees.

The evolution of planning methods mirrors a broader shift in the research problem itself. As shown in Figure 2, early work emphasized reachability and route length in known static environments, whereas later studies focused on dynamic obstacles and real-time collision avoidance. Subsequent frameworks incorporated vessel dynamics, COLREGs, environmental disturbances, and multi-vessel interactions. More recent research has turned to partial observability, model mismatch, cross-scenario generalization, interpretability, safety verification, and real-vessel deployment. Evaluation has therefore expanded beyond path length and computation time to include safety margins, rule compliance, manoeuvring feasibility, energy efficiency, robustness, and deployment readiness. The central challenge is no longer simply to generate a feasible path, but to sustain safe, compliant, executable, and verifiable decisions under realistic maritime traffic conditions.

Existing reviews have approached ship path planning from several directions. Broad surveys have classified USV autonomy, path-planning architectures, and collision-avoidance systems [1,2,5,9]. Others have compared search, sampling, optimization, and local planning algorithms [7,8,11,18], or focused on collision avoidance, COLREGs compliance, multi-vessel coordination, and machine-learning-based decision-making [19,20,21,22,23,24]. These studies establish a valuable foundation, but three gaps remain.

First, global route planning, local collision avoidance, trajectory generation, and motion control are often reviewed in isolation, leaving their functional hierarchy and module interfaces unclear. Second, model-based and learning-based methods are commonly assessed using different vessel models, traffic scenarios, and metrics, which limits meaningful comparison. Third, deployment-related issues—including real-world data, simulation platforms, digital twins, Sim-to-Real transfer, full-scale trials, and safety verification—remain fragmented across the literature. A unified framework linking algorithm design, scenario-based evaluation, and engineering deployment is therefore still lacking.

Here, ship path planning is treated as the intermediate decision layer connecting perception, navigational decision-making, trajectory generation, and control execution. The review covers global route generation in known environments, local replanning and collision avoidance in dynamic traffic, and trajectory generation when it is coupled directly to planning. Pure path-following controllers that neither update the route nor make collision-avoidance decisions are outside the main scope. The analysis centres on four questions: how maritime environments are represented, how planning decisions are generated, how vessel dynamics and navigational rules are incorporated, and how methods progress from simulation-based validation to reliable operation at sea.

This review develops a unified framework that organizes ship path-planning methods by planning scope and decision basis, clarifying the roles and complementarities of model-based, learning-driven, and hybrid approaches. We compare representative studies across objective design, state representation, policy optimization, rule integration, environmental uncertainty, validation platforms, and operational settings, and identify the design choices that most strongly affect performance and deployability. The analysis further exposes persistent barriers in rule formalization, multi-vessel interaction, standardized evaluation, cross-scenario generalization, decision trustworthiness, and full-scale validation, and outlines emerging directions in inverse reinforcement learning, game-theoretic reasoning, digital twins, transfer and meta-learning, world models, and safe hybrid planning.

The remainder of this review is organized as follows. Section 2 positions the present work relative to previous reviews. Section 3 introduces autonomy levels, the literature-selection methodology, and the formulation of ship path planning. Section 4 reviews model-based priors and constraints, value-based and actor–critic methods, multi-agent learning, and hybrid DRL frameworks. Section 5 discusses key design choices, unresolved challenges, and engineering deployment, and Section 6 concludes the review.

2. Related Work

Reviews of ship path planning have expanded from algorithm-centred taxonomies to broader assessments of autonomous-shipping safety, reliability, and technological development. Chaal et al. [25] mapped autonomous-ship research from risk, safety, and reliability perspectives. Li et al. [26] examined the thematic structure of MASS risk and reliability studies published between 2015 and 2022, while Xue et al. [27] analysed machine-learning developments in autonomous-shipping safety. These bibliometric studies reveal major research communities and thematic trends, but provide limited comparison of path-planning mechanisms, constraint handling, and algorithmic performance.

Figure 3 shows the annual distribution of the reviewed literature from 2010 to 2026. Publication activity increased markedly after 2020, from 10 studies in 2021 to 13 in 2022, 18 in 2023, and 25 in 2024, followed by 23 in 2025, reflecting growing attention to learning-based collision avoidance, multi-vessel interaction, generalization, and deployment-oriented navigation.

A second group of reviews has focused on specific operating conditions. Enevoldsen et al. [28] examined autonomous ferries and harbour buses, emphasizing the coupling of collision avoidance, grounding prevention, and regulatory compliance. Ding et al. [29] reviewed perception degradation, risk assessment, and collision-avoidance decisions in fog, while Jovanović et al. [30] surveyed enabling technologies and research gaps in autonomous shipping. These studies demonstrate the influence of vessel type, environment, and mission profile, but their scenario-specific scope limits comparison across open seas, ports, inland waterways, and dense traffic.

Operational realism has also made testing and validation a central concern. Beyond route length, computation time, and collision counts, autonomous-vessel planners must be evaluated for COLREGs compliance, navigational acceptability, perception robustness, and resilience to uncertainty. Hagen et al. [31] proposed a framework for evaluating safety and regulatory compliance, Clement et al. [32] introduced a simulator for hybrid-navigation safety and acceptability, and Vekinis and Perantonis [33] developed an environment for COLREG-compliant USV navigation and maritime-object detection. Raza et al. [34] further proposed an integrated digital-twin architecture. These studies shift evaluation towards system-level validation, although differences in vessel models, traffic generation, disturbances, rule assessment, and metrics still hinder reproducible comparison.

Figure 4 shows the interdisciplinary distribution of publication venues. Ocean Engineering contributes the largest share, followed by the Journal of Marine Science and Engineering, conference proceedings, and preprint platforms. The remaining studies span artificial intelligence, robotics, control, transportation, and maritime engineering, reflecting the convergence of algorithmic, regulatory, and operational perspectives.

Multi-vessel interaction extends bilateral collision avoidance to coupled decision-making, because manoeuvring responsibilities, coordinated actions, and policy responses are interdependent. Such problems are commonly formulated as multi-agent decision processes using communication, shared rewards, or game-theoretic structures. Wei and Kuo [35] developed COLREGs-compliant multi-agent collision avoidance, Niu et al. [36] learned coordinated policies from multi-vessel interaction data, and Verma and Samvedi [37] considered cooperative avoidance in mixed traffic. General studies have examined competition, cooperation, and equilibrium behaviour through game theory and multi-agent reinforcement learning [38], while maritime applications have adopted multi-agent reinforcement learning for cooperative collision avoidance and path following [39]. However, most methods remain focused on local conflict resolution, with global route consistency, heterogeneous behaviour, scalable coordination, and responsibility transfer still insufficiently addressed.

A parallel direction raises planning from motion generation to semantic reasoning and mission interpretation. LLMs may translate natural-language objectives, maritime knowledge, and navigation rules into structured recommendations. Pei et al. [40] assessed the ship-handling knowledge of LLMs, Sanchez-Heres et al. [41] examined COLREGs interpretation, Christensen et al. [42] developed conversational mission planning and execution, Agyei et al. [43,44] explored COLREGs-guided and risk-aware decision support, and Din et al. [45] applied an LLM to USV mission planning. LLMs may therefore support mission-intent interpretation, rule reasoning, and human–machine interaction, but concerns over consistency, hallucination, latency, and verifiable safety limit their use as low-level planners. Their near-term role is more plausibly supervisory, above explicitly constrained planning algorithms.

Behavioural acceptability and interpretability are also becoming important. Higaki and Hashimoto [46] used generative adversarial imitation learning to reproduce expert-like routes, Zheng et al. [47] combined rule guidance with vision-based supervised learning, and Rong et al. [48] introduced human-like attention to prioritize high-risk vessels and regions. Other work incorporated implicit human preferences into multi-agent policy fine-tuning [49] and used subtask-value decomposition and policy visualization to explain avoidance behaviour [50]. These methods broaden evaluation towards maritime-practice consistency and comprehensibility, but human likeness remains difficult to define, collect, and verify, and post hoc explanations do not necessarily imply safer decisions.

Deployment further requires generalization beyond the training distribution. Wang et al. [51] used knowledge transfer for more efficient and safer collision avoidance, while Jiang et al. [52] applied meta-reinforcement learning to adaptation under unknown currents. Digital twins link predictive models, simulation, and operational data for situational awareness and safe navigation [53,54], and supervised learning has been combined with digital-twin environments to generate representative planning data [55]. These developments extend evaluation from a fixed simulator towards robustness across vessels, waterways, and disturbance regimes. However, distribution shift, simulator bias, and closed-loop adaptation using real-vessel data are rarely quantified, underscoring the need for reproducible Sim-to-Real benchmarks.

Planning safety is also constrained by perception. Sensor noise, occlusion, missed detections, and partial observability distort risk estimates and motion predictions. Larsen et al. [56] combined risk-aware convolutional perception with DRL, Lee et al. [57] evaluated reactive avoidance under sensor noise, Fan et al. [58] used memory for COLREGs-compliant decisions with limited knowledge, Wu et al. [59] developed risk-aware mapless navigation, and Jin et al. [60] combined predictive avoidance with disturbance-aware control. These studies highlight the importance of uncertainty, memory, and risk representation, although most rely on predefined noise models and neglect correlated errors, abrupt manoeuvres, and long-horizon risk accumulation.

Planning is increasingly embedded in mission and control architectures. Jin et al. [61] applied multi-task reinforcement learning to formation control under environmental disturbances, Wang et al. [62] addressed moving-target enclosure in cluttered environments, Zhang et al. [63] developed DRL-based multi-USV task planning, and Yin and Xiang [64] jointly formulated target assignment and path planning. Tao et al. [65] further integrated guidance, anticipatory collision avoidance, and motion control. These studies position path planning as a decision layer coupled to mission management, cooperative control, and vessel dynamics, rather than an isolated route optimizer. However, interfaces between these layers and the propagation of interpretable safety constraints from mission objectives to executable actions remain insufficiently standardized.

Recent studies have also addressed complex-water MASS collision avoidance, navigation under the IALA Maritime Buoyage System, PPO-based decisions in restricted waters, real-time avoidance among multiple obstacle vessels, and multi-USV planning under limited perception [66,67,68,69,70]. Together, they confirm that ship path planning is no longer an isolated route-generation problem, but a system-level challenge spanning environmental modelling, rule representation, interactive decision-making, control execution, and validation. Unlike reviews that isolate collision avoidance, reinforcement learning, or particular operating scenarios, this review links methodological paradigms, design choices, application environments, and deployment barriers to identify how planning methods have evolved and what is required for reliable autonomous navigation.

3. Framework for Reviewing and Evaluating Existing Methods

Rather than catalogue algorithms in isolation, this review evaluates ship path-planning methods through three complementary lenses: autonomy level, literature selection, and problem formulation. Autonomy level defines the responsibility assigned to the planner, from providing advice to human operators to supporting a closed navigation loop spanning perception, decision-making, collision avoidance, and control. The literature-selection procedure determines the scope, representativeness, and reproducibility of the evidence base. Problem formulation, in turn, defines the dimensions used for comparison, including navigable-space representation, dynamic-obstacle handling, regulatory compliance, vessel dynamics, real-time performance, and deployment-oriented validation. Together, these perspectives provide a consistent basis for assessing the functional limits and engineering potential of existing methods.

3.1. Levels of Autonomy

The role of path planning depends strongly on the degree of vessel autonomy. The IMO framework, as summarized by Felski and Zwolak [71], classifies maritime autonomous surface ships according to the extent of human involvement in navigation and control, while the Norwegian Forum for Autonomous Ships (NFAS) describes autonomy as a progressive transfer of authority from onboard crews to shipborne systems [72]. Although the two schemes differ in detail, both imply that increasing autonomy places greater responsibility on onboard systems for perception, planning, collision avoidance, and control.

For path planning, autonomy therefore defines the boundary between decision support and autonomous responsibility. At Levels 1 and 2, planners mainly provide route recommendations, risk warnings, and collision-avoidance advice to onboard crews or remote operators. Their outputs must consequently be interpretable, auditable, and easy to override. Ramos et al. [3] showed that autonomous collision avoidance must be designed around human supervision, judgement, and takeover behaviour, while Burmeister and Constapel [19] identified formal rule representation and regulatory acceptability as prerequisites for deployment. Table 2 summarizes how the role and methodological requirements of path planning change across autonomy levels.

At Levels 3 and 4, path planning becomes a closed-loop decision module rather than an advisory tool. Without continuous human intervention, the vessel must integrate electronic navigational charts, onboard sensing, Automatic Identification System (AIS) data, and target-vessel states to update global routes, avoid local conflicts, enforce navigational rules, and assess trajectory feasibility in real time. Negenborn et al. [4] emphasized that deployment depends not only on algorithmic performance, but also on safety, liability, regulation, and system integration. Alamoush and Ölçer [5] likewise showed that perception, decision-making, planning, control, and communication must operate as a coordinated architecture. High-autonomy planners must therefore combine real-time collision avoidance with regulatory consistency, robustness, generalization, and verifiable safety.

Autonomy level consequently provides a useful basis for comparing planning paradigms. Search- and sampling-based methods are most naturally suited to global reference-route generation, whereas geometry- and rule-based methods provide interpretable local guidance. Optimization methods support receding-horizon trajectory generation under explicit constraints, while learning-driven and hybrid approaches are better positioned to address the interactive and uncertain conditions associated with Levels 3 and 4. As autonomy increases, however, evaluation must extend beyond route length, computation time, and collision-avoidance success to include human takeover, rule compliance, fault degradation, validation procedures, and deployment readiness [9,17,31].

Figure 5 summarizes this transition. With increasing autonomy, path planning shifts from human-centred navigation support to system-level closed-loop decision-making, and the evaluation focus moves from interpretability and ease of intervention towards real-time safety, regulatory compliance, robust generalization, and verifiable deployment.

3.2. Methodology for Literature Selection

A systematic review requires a transparent and reproducible evidence-selection process. Rather than focusing on a single algorithm family, this review covers the full spectrum of ship path planning, from search, geometric, rule-based, and optimization methods to learning-driven and hybrid frameworks. The corpus therefore includes both foundational studies and recent work on deep reinforcement learning, multi-agent coordination, digital twins, simulation-based validation, and engineering deployment.

The Web of Science Core Collection, Scopus, and Google Scholar were searched for publications from 2010 to 2026, with emphasis on the most recent five years. Search terms were grouped into three sets: research platforms (“ship,” “vessel,” “USV,” “ASV,” and “MASS”), planning tasks (“path planning,” “route planning,” “trajectory planning,” “collision avoidance,” and “obstacle avoidance”), and methodological approaches (“A*,” “RRT,” “velocity obstacle,” “COLREGs,” “model predictive control,” “optimization,” “deep reinforcement learning,” “multi-agent reinforcement learning,” “hybrid planning,” “digital twin,” and “safety verification”). Boolean combinations of these terms were used to capture differences in terminology across route planning, collision avoidance, and trajectory-generation research.

The initial search returned approximately 512 candidate publications. After duplicate removal and screening of titles, abstracts, keywords, and full texts, 112 studies directly relevant to ship path planning were retained. Nine additional publications were included to establish the theoretical origins of key algorithms, yielding a final corpus of 121 references.

Screening was conducted in two stages. The first excluded studies centred on non-surface platforms, sensor hardware, communication protocols, isolated path following, or task management without an explicit planning component. Full texts were then retained when they: (i) considered a USV, autonomous surface vessel, MASS, or related surface-ship platform; (ii) addressed global routing, local replanning, dynamic collision avoidance, trajectory generation, or integrated planning and control; (iii) described the environment, constraints, method, experimental setting, or evaluation criteria with sufficient clarity; and (iv) provided evidence relevant to methodological comparison, research challenges, or future directions.

The retained studies were organized using a three-level structure: methodological paradigm, research problem, and validation setting. At the first level, methods were classified as search- and sampling-based, geometry- and rule-based, optimization-based, learning-driven, or hybrid and hierarchical. At the second, they were compared in terms of objective and reward design, state representation, exploration and optimization, safety-rule integration, environmental disturbances, and multi-vessel interaction. At the third, studies were grouped by validation setting, including simulation platforms, digital twins, ports and inland waterways, open-water trials, and full-scale experiments. This structure emphasizes functional roles, interfaces, and applicability rather than algorithm names alone.

Inclusion decisions jointly considered relevance, methodological representativeness, publication quality, experimental completeness, and influence. Classical studies were retained to establish the foundations of A*, RRT, velocity obstacles, model predictive control, COLREGs-based reasoning, and reinforcement learning. For emerging topics—including DRL, multi-agent learning, predictive safety filtering, digital twins, human-like decision-making, and LLM-assisted planning—priority was given to studies that captured recent developments while remaining directly relevant to path planning [21,23,25,27]. Conference papers and preprints were included only where journal coverage was limited and were not treated as equivalent to fully validated engineering evidence. Table 3 summarizes the inclusion and exclusion criteria.

Figure 6 presents the five-stage workflow: database retrieval, keyword combination, preliminary screening, full-text assessment, and final classification. The resulting corpus supports the subsequent analysis of both methodological evolution and the cross-cutting design choices that govern performance and deployability.

3.3. Fundamentals of Ship Path Planning

Environmental representation determines what the planner can perceive, predict, and constrain. Static navigability is defined by coastlines, bathymetry, reefs, fairway boundaries, restricted zones, and infrastructure, whereas dynamic traffic is described by the positions, velocities, headings, and inferred intentions of surrounding vessels. Geometric and probabilistic methods express collision risk through ship domains, velocity obstacles, reciprocal velocity obstacles, and sampling-based risk assessment [75,76,77]. Optimization methods encode safety margins, route efficiency, energy use, smoothness, and regulatory requirements as objectives or constraints. Learning-based methods instead map own-ship states, target-vessel information, local observations, and interaction history to actions, with behaviour shaped through reward design and embedded rule priors [14,16]. The central modelling question is therefore not only how to represent free space, but how to retain the uncertainty, interaction structure, and dynamic feasibility required for reliable decisions.

Safety is the primary criterion in ship path planning. Common risk measures include the distance and time to the closest point of approach (DCPA and TCPA), ship domains, and collision probability. Tam and Bucknall [78] established an early framework for collision-risk assessment, while Huang et al. [6] organized collision avoidance around motion prediction, conflict detection, and conflict resolution. For autonomous vessels, however, safety extends beyond collision avoidance to grounding prevention, fairway and boundary compliance, and exclusion from restricted waters. A credible planner must also preserve adequate safety margins under uncertain observations, model error, and unexpected target-vessel manoeuvres.

Regulatory compliance further distinguishes maritime planning from generic robotic obstacle avoidance. The COLREGs assign manoeuvring responsibilities in head-on, crossing, and overtaking encounters, constraining not only whether a vessel acts but also when, in which direction, and with what magnitude. Kuwata et al. [75] embedded COLREGs within a velocity-obstacle framework, providing an early example of rule-constrained geometric avoidance. Du et al. [79] developed a compliant collision-alert system for stand-on vessels, whereas Wróbel et al. [10] highlighted the ambiguity and context dependence of many COLREGs provisions. A valid plan must therefore be more than collision-free: it must produce timely, observable, and operationally acceptable manoeuvres consistent with both formal rules and established maritime practice.

Vessel dynamics ultimately determine whether a planned route can be executed. Large inertia, underactuation, and hydrodynamic coupling can render a geometrically feasible path dynamically unattainable. Planners must therefore account for turning radius, rudder and speed limits, heading-rate constraints, and control continuity. Fossen [80] provides the theoretical foundation for marine-craft modelling and control. Building on this basis, Tsolakis et al. [81] incorporated traffic rules into model-predictive trajectory optimization, while Vaaler et al. [17] used predictive safety filters to constrain reinforcement-learning actions. These studies reinforce a central principle: for high-autonomy vessels, planning, control, and safety verification cannot be designed independently.

Uncertainty further complicates this coupling. Changes in vessel intent, wind–wave–current disturbances, sensor noise, and partial observability can rapidly invalidate a nominal plan. Model-based approaches address these effects through prediction, receding-horizon optimization, and robust constraints, whereas learning-based methods use recurrent networks, memory, attention, domain randomization, and risk-sensitive objectives to improve adaptation. Lee et al. [57] examined reactive collision avoidance under varying sensor-noise levels. Fan et al. [58] introduced memory-enhanced DRL for limited environmental knowledge, and Wu et al. [59] developed risk-aware mapless navigation for uncertain and congested waters. Robust planning therefore depends not only on the decision algorithm, but also on how uncertainty, temporal context, and risk are represented under incomplete information.

Based on this formulation, we evaluate ship path-planning methods along six dimensions: safety, efficiency, regulatory compliance, manoeuvring feasibility, real-time performance, and robustness. Safety covers collision, grounding, and boundary-violation risks; efficiency includes route length, travel time, and energy use; regulatory compliance concerns COLREGs and local navigation rules; manoeuvring feasibility reflects curvature, speed variation, and control limits; real-time performance measures the ability to update decisions online; and robustness captures sensitivity to sensor error, behavioural uncertainty, and environmental disturbances. Section 4 examines how different methodological paradigms address these requirements, while Section 5 compares their design trade-offs and deployment readiness.

Figure 7 summarizes this problem structure. Environmental observations, vessel states, and mission and regulatory constraints provide the principal inputs. Global planning, local replanning, and trajectory generation then produce executable references, which are assessed against the six evaluation dimensions defined above.

4. Ship Path Planning with Emphasis on DRL-Based Decision-Making

Ship path planning has progressed from static route generation to dynamic collision avoidance, rule-compliant manoeuvring, and multi-vessel interaction. This shift has moved the field beyond purely model-based solutions towards decision frameworks that combine structured maritime knowledge with data-driven policies. Conventional methods have not been displaced by deep reinforcement learning (DRL); instead, they increasingly provide priors, constraints, and safety mechanisms within learning-based systems. The resulting methodological landscape spans model-informed planning, value-based decision-making, policy optimization, multi-agent coordination, and safety-oriented hybrid control.

This section therefore organizes existing methods by their functional role in autonomous navigation rather than by the historical development of reinforcement-learning algorithms. We first examine how search, geometric, rule-based, and optimization methods supply route priors, risk representations, and explicit safety boundaries for DRL. We then review value-based methods for discrete planning decisions and actor–critic approaches for continuous manoeuvring. Finally, we discuss multi-agent DRL for coupled vessel interactions and hybrid frameworks that combine adaptive policies with rule enforcement, predictive safety filters, or model-based control. Figure 8 summarizes the relationships among these methodological families and their roles within the ship path-planning pipeline.

4.1. Model-Based Methods as Planning Priors and Safety Constraints

As ship path planning has expanded from static routing to dynamic collision avoidance, regulatory compliance, and closed-loop autonomy, model-based methods have shifted from stand-alone planners to structural components within learning-based systems. Search and sampling methods provide global route guidance, geometric methods encode relative-motion risk, rule-based methods formalize COLREGs responsibilities, and optimization methods enforce dynamic feasibility through receding-horizon constraints or safety filters. Their role is therefore increasingly to shape, constrain, and verify learned policies rather than to replace them.

At the global level, search- and sampling-based methods exploit electronic navigational charts, coastlines, bathymetry, restricted waters, and static obstacles to generate low-risk reference routes. Singh et al. [73] incorporated dynamic obstacles and ocean-current costs into constrained A* search, while Zhao et al. [74] separated global route generation from local collision avoidance within a hierarchical architecture. In complex traffic, the main contribution of these methods is thus not to solve the entire navigation problem independently, but to preserve mission-level consistency and provide stable spatial priors for adaptive local policies.

At the local decision and execution levels, geometric, rule-based, and optimization models are commonly integrated with DRL through state augmentation, reward shaping, action-space restriction, or post-policy safety correction. Artificial potential fields encode goal attraction and obstacle repulsion; velocity-obstacle, reciprocal-velocity-obstacle, and dynamic-window methods identify unsafe motions or restrict admissible actions; and COLREGs modules impose encounter responsibilities and preferred manoeuvring directions. Model predictive control and predictive safety filters can then correct policy outputs that violate dynamic or safety constraints [12,13,15,16,17]. These components reduce the effective search space while embedding maritime knowledge, rule compliance, and feasibility directly into the decision process. Table 4 summarizes the main functions of model-based components in DRL-assisted ship path planning.

Model-based priors generally improve sample efficiency, rule compliance, and action feasibility, but their benefits depend on the fidelity of the underlying models. Overly restrictive geometric or regulatory constraints can suppress adaptation to atypical encounters, while modelling errors may propagate through state features, reward terms, or safety filters. Moreover, most studies compare complete hybrid systems with pure DRL baselines, providing limited evidence on the individual contributions of route priors, risk representations, rule modules, and safety filters. Systematic ablation studies and uncertainty-aware interface design are therefore needed to determine when model-based structure improves learned policies and when it instead introduces bias or brittleness.

4.2. Value-Based DRL for Discrete Path-Planning Decisions

Value-based DRL is well suited to discrete planning decisions, such as selecting heading changes, rudder-angle levels, speed commands, or local waypoints. Its explicit action-value formulation also facilitates integration with occupancy grids, collision-risk maps, and COLREGs encounter classifications. Most maritime applications build on deep Q-networks (DQNs), with extensions including Double DQN, Dueling DQN, prioritized experience replay, parameter noise, and learned environment models to improve value estimation, representation capacity, exploration, and sample efficiency.

A DQN approximates the state–action value function with a neural network and stabilizes learning through experience replay and a separate target network, as introduced by Mnih et al. [83]. In ship collision avoidance, the state typically combines own-ship motion, relative target-vessel states, obstacle grids, or local risk maps, while the action is represented by a discrete heading or rudder adjustment. The temporal-difference target is given by

$$y_t=r_t+\gamma \underset{a^{\prime }}{max}{Q}_{{\theta }^{-}}\left(s_{t+1},a^{\prime }\right),$$

(1)

where $r_t$ denotes the immediate reward, $\gamma$ is the discount factor, and ${\theta }^{-}$ represents the parameters of the target network. The DQN loss function is given by

$$L(\theta )=\mathbb{E}\left[{\left(y_t-Q_{\theta }(s_t,a_t)\right)}^2\right],$$

(2)

where $\theta$ denotes the parameters of the online Q-network.

Although DQN stabilizes discrete value learning through experience replay and a target network, its performance remains sensitive to action-space discretization and value overestimation. Double DQN mitigates this bias by decoupling action selection from target evaluation, whereas Dueling DQN decomposes the action value into state-value and action-advantage terms. This decomposition is particularly useful when many actions have similar consequences, as in low-risk, long-range navigation [84,85]. Woo and Kim [86] demonstrated the feasibility of Double and Dueling DQN in representative USV encounters. Xu et al. [87] embedded COLREGs constraints into discrete collision-avoidance learning, while Gao et al. [88] combined Dueling DQN with prioritized experience replay for AIS-driven autonomous navigation. The corresponding Double DQN target and Dueling DQN decomposition are

$$y_t^{\mathrm{DDQN}}=r_t+\gamma Q_{{\theta }^{-}}\left(s_{t+1},\underset{a^{\prime }}{arg\; max}{Q}_{\theta }\left(s_{t+1},a^{\prime }\right)\right),$$

(3)

and

$$Q(s,a)=V(s)+\left(A(s,a)-{\displaystyle \frac{1}{|\mathcal{A}|}}\sum _{a^{\prime }}A(s,a^{\prime })\right),$$

(4)

where $V(s)$ denotes the state-value function, $A(s,a)$ is the action-advantage function, and $|\mathcal{A}|$ denotes the number of actions in the discrete action space.

In maritime applications, value-based architectures are increasingly combined with mechanisms that improve sample use and inject structured navigational knowledge. Gao et al. [88] paired Dueling DQN with prioritized experience replay for AIS-driven MASS navigation, while Yang et al. [89] integrated Dueling DQN with velocity obstacles to couple value estimation with geometric risk prediction.

Subsequent developments follow two main directions. The first improves exploration and data efficiency: Noisy DQN introduces learnable parameter noise for state-dependent exploration [90], whereas Dyna augments model-free learning with simulated transitions generated by an environment model [91]. Such mechanisms are particularly relevant to maritime tasks, where hazardous encounters are difficult and costly to collect. The second direction embeds domain knowledge into discrete policies. Artificial potential fields provide goal guidance, velocity obstacles anticipate unsafe motions, COLREGs constrain manoeuvring responsibilities, and recurrent memory captures longer-term interactions [12,66,89,92,93]. Value-based ship planning has therefore progressed from learning basic avoidance actions to improving sample efficiency, risk awareness, temporal reasoning, and regulatory consistency within a discrete action space.

Table 5 compares the algorithmic structures, decision outputs, application settings, and principal contributions of representative value-based methods.

Value-based methods are most effective when the state representation is structured, the decision rules are well defined, and the action set is finite. They are therefore well suited to selecting heading changes, speed levels, local waypoints, or high-level avoidance commands. Their principal limitation is action discretization. Coarse action sets cannot represent smooth vessel manoeuvres, whereas fine discretization enlarges the search space and makes value estimation less stable.

Evidence also remains concentrated on predefined encounter types and fixed action partitions. Few studies systematically examine how action resolution affects manoeuvring feasibility, learning stability, or generalization across traffic conditions. Value-based methods are therefore better positioned as high-level decision modules within hierarchical architectures than as direct controllers of continuous rudder and propulsion commands. This limitation motivates actor–critic methods, which optimize continuous manoeuvring actions directly.

4.3. Policy-Gradient and Actor–Critic DRL for Continuous Ship Control

Policy-gradient and actor–critic methods are better suited than value-based approaches to the continuous actions required for ship manoeuvring, including rudder angle, desired heading, speed, thrust, and control moment. They therefore avoid the resolution–complexity trade-off introduced by action discretization. Maritime applications have primarily adopted proximal policy optimization (PPO), asynchronous advantage actor–critic (A3C), deep deterministic policy gradient (DDPG), twin delayed DDPG (TD3), and soft actor–critic (SAC). These methods differ in whether learning is on- or off-policy, how policy updates are constrained, how experience is reused, and how exploration and value-estimation errors are controlled [95,96,97,98,99].

PPO has become particularly prominent in maritime planning because its clipped updates provide stable training and accommodate modular state, reward, and safety designs. Early studies applied PPO to COLREGs-constrained collision avoidance and integrated path-following and avoidance tasks [82,100]. Subsequent work introduced rule-informed risk features and collision-grid representations for complex traffic [101,102]. Other extensions coupled PPO with dynamic-window priors, evaluated robustness to perception noise, or constrained policy outputs using predictive safety filters [16,17,57]. Wang et al. [103] further regulated exploration through cognitive entropy to adapt the exploration–exploitation balance during collision avoidance. These studies indicate that PPO’s main advantage lies in stable and extensible policy optimization; regulatory compliance, dynamic feasibility, and safety still depend on the information, constraints, and supervisory mechanisms embedded around the policy. Its clipped surrogate objective is

$$L^{\mathrm{clip}}(\theta )={\mathbb{E}}_t\left[min\left(r_t(\theta )A_t,clip\left(r_t(\theta ),1-ϵ,1+ϵ\right)A_t\right)\right],$$

(5)

where $r_t(\theta )$ denotes the probability ratio between the updated and previous policies, $A_t$ is the estimated advantage, and $ϵ$ specifies the clipping range.

A3C increases data-collection throughput by training multiple asynchronous workers in parallel. Xie et al. [104] applied this architecture to multi-vessel collision avoidance. In the literature reviewed here, however, A3C appears less frequently than PPO, TD3, and SAC, partly because asynchronous updates can be sensitive to reward scaling, gradient variance, and coordination among workers.

DDPG is more directly suited to tightly coupled planning and control because its deterministic actor generates continuous rudder angles, thrust commands, or control moments. Xu et al. [105] applied DDPG to real-time COLREGs-compliant planning and collision avoidance, while Lou et al. [106] incorporated motion prediction and further validated the approach in field experiments. The method nevertheless remains sensitive to exploration noise, hyperparameters, and critic error. In safety-critical encounters, overestimated action values can translate directly into hazardous control commands. Reliable DDPG-based navigation therefore depends on carefully normalized states and rewards, adequate exploration, and explicit feasibility or safety constraints. The critic is trained using the temporal-difference target

$$y_t=r_t+\gamma Q_{{\varphi }^{\prime }}\left(s_{t+1},{\mu }_{{\theta }^{\prime }}\left(s_{t+1}\right)\right),$$

(6)

and the critic loss is defined as

$$L_Q(\varphi )=\mathbb{E}\left[{\left(Q_{\varphi }(s_t,a_t)-y_t\right)}^2\right].$$

(7)

The actor is updated by maximizing the action value estimated by the critic:

$$J_{\mu }(\theta )={\mathbb{E}}_s\left[Q_{\varphi }\left(s,{\mu }_{\theta }(s)\right)\right].$$

(8)

TD3 improves DDPG by introducing twin critics, delayed policy updates, and target-policy smoothing, and has consequently become an important method for continuous-control-based ship collision avoidance. Its target value adopts the smaller estimate produced by the two target critics:

$$y_t=r_t+\gamma \underset{i=1,2}{min}{Q}_{{\varphi }_i^{\prime }}\left(s_{t+1},{\tilde{a}}_{t+1}\right),$$

(9)

where the smoothed target action is given by

$${\tilde{a}}_{t+1}={\mu }_{{\theta }^{\prime }}\left(s_{t+1}\right)+\epsilon ,\epsilon \sim clip\left(\mathcal{N}(0,\sigma ),-c,c\right).$$

(10)

By combining twin critics with delayed policy updates and target-policy smoothing, TD3 reduces the value overestimation and unstable updates that affect DDPG. Sun et al. [107] coupled Meta-TD3 with COLREGs-based planning to improve adaptation across encounter scenarios. Waltz et al. [108] developed a hierarchical framework for inland-waterway navigation, using LSTM-TD3 for planning and path following under AIS-replay conditions. Rong et al. [48] further integrated recurrent units and multi-head attention with TD3 to focus policy decisions on high-risk vessels during complex encounters. These studies extend TD3 from continuous control towards temporal reasoning, cross-scenario adaptation, and attention-guided decision-making. Twin critics, however, mitigate estimation bias rather than enforce navigational safety. Without adequate coverage of abnormal vessel behaviour or explicit rule and safety constraints, a numerically stable critic may still support unsafe actions.

SAC addresses exploration more directly by optimizing a maximum-entropy objective. The policy is trained to maximize cumulative return while retaining sufficient stochasticity, which can improve exploration and robustness in uncertain maritime environments. Its objective is expressed as

$$J(\pi )={\mathbb{E}}_{\pi }\left[\sum _t{\gamma }^t\left(r_t+\alpha \mathcal{H}\left(\pi (·\mid s_t)\right)\right)\right],$$

(11)

where $\alpha$ controls the contribution of entropy and $\mathcal{H}\left(\pi (·\mid s_t)\right)$ measures policy uncertainty at state $s_t$. Entropy regularization allows SAC to maintain broader exploration than deterministic actor–critic methods, reducing premature convergence and improving adaptation to uncertain or disturbed environments. Zhao et al. [109] applied a priority-sampling SAC framework to continuous USV navigation in port waters, incorporating IALA buoyage rules and wind–wave–current disturbances. Jin et al. [110] coupled SAC with a world model that learns latent environmental dynamics, thereby improving navigation under external disturbances. These studies position SAC as a suitable foundation for combining policy learning with world models, domain randomization, and disturbance-aware training to strengthen generalization across complex maritime conditions.

Table 6 compares representative actor–critic methods in terms of algorithmic structure, action output, operating environment, and principal contribution.

Current evidence does not support a universal ranking of PPO, DDPG, TD3, and SAC for ship path planning. Reported results are confounded by differences in vessel models, observations, action spaces, reward design, training budgets, and encounter scenarios; cumulative returns are therefore not directly comparable across studies.

Broadly, PPO favours stable updates and straightforward integration with rule-based rewards, risk features, and safety filters. TD3 improves deterministic continuous control through twin critics, delayed policy updates, and target-policy smoothing. SAC combines off-policy sample reuse with entropy-driven exploration, whereas DDPG is structurally simpler but more sensitive to exploration noise, reward scaling, hyperparameters, and critic bias.

Meaningful comparisons require standardized vessel dynamics, observations, actions, rewards, training budgets, and safety constraints. Evaluation should extend beyond cumulative return to collision and success rates, COLREGs compliance, minimum separation distance, control smoothness, computational cost, and generalization across vessel types, traffic densities, disturbances, and unseen encounters.

4.4. Multi-Agent and Hybrid DRL for Complex Maritime Interactions

As maritime planning expands from single-vessel avoidance to multi-vessel encounters, formation coordination, and mixed traffic, single-agent DRL becomes increasingly inadequate. Surrounding vessels are not passive obstacles but decision-making agents whose intentions and manoeuvres alter the collision risk and feasible actions of others. The problem is therefore intrinsically coupled, motivating a shift towards multi-agent deep reinforcement learning (MADRL) and hybrid frameworks that combine learned coordination with explicit safety mechanisms.

MADRL must address three related challenges: non-stationarity, partial observability, and coordination consistency. Centralized training with decentralized execution (CTDE) has consequently become a dominant paradigm. During training, global states or joint actions support more stable value estimation; during execution, each vessel acts independently from local observations. QMIX is a representative value-decomposition method for cooperative multi-USV tasks, factorizing a global action-value function into agent-specific utilities while preserving decentralized action selection [111]. Its monotonicity constraint is expressed as

$$Q_{\mathrm{tot}}(\mathit{s},\mathit{a})=f\left(Q_1,Q_2,\dots ,Q_n,\mathit{s}\right),{\displaystyle \frac{\partial Q_{\mathrm{tot}}}{\partial Q_i}}\ge 0,$$

(12)

where $Q_{\mathrm{tot}}$ is the joint action-value function, $Q_i$ is the utility of agent i, and $\mathit{s}$ and $\mathit{a}$ denote the global state and joint action. The monotonic constraint aligns decentralized action selection with the cooperative objective: increasing an individual utility cannot reduce the estimated global value [111].

Maritime applications increasingly use this multi-agent perspective to model coupled avoidance decisions. Wei and Kuo [35] developed a COLREGs-compliant framework for multi-vessel collision avoidance, showing that cooperative policies can account for both separation requirements and encounter responsibilities. Niu et al. [36] learned coordinated avoidance strategies directly from multi-vessel interaction data, while Verma and Samvedi [37] extended cooperative planning to mixed traffic. These studies move MADRL beyond homogeneous fleets towards environments containing vessels with different dynamics, policies, and levels of autonomy. However, most evaluations still involve limited fleet sizes and structured encounters, leaving scalability and robustness to heterogeneous behaviour unresolved.

Value decomposition is primarily suited to cooperative tasks with discrete action spaces. Continuous maritime control instead favours multi-agent actor–critic methods, which can directly generate rudder angles, velocities, thrust commands, and other continuous manoeuvring variables. Multi-agent deep deterministic policy gradient (MADDPG) assigns each vessel an independent actor while training a centralized critic using the joint observations and actions of all agents. By incorporating global interaction information during training, the centralized critic mitigates the non-stationarity caused by the simultaneous adaptation of multiple policies, while decentralized actors remain executable using only local observations [113]. For agent i, the centralized action-value estimate can be expressed as

$${\widehat{Q}}_i=Q_i^{\pi }\left(\mathit{o},\mathit{a}\right),\mathit{a}=\left[{\mu }_1(o_1),{\mu }_2(o_2),\dots ,{\mu }_N(o_N)\right],$$

(13)

where $\mathit{o}=[o_1,o_2,\dots ,o_N]$ denotes the joint observation, $\mathit{a}=[a_1,a_2,\dots ,a_N]$ is the joint action, ${\mu }_i(o_i)$ is the decentralized actor of agent i, and ${\widehat{Q}}_i$ is the centralized action-value estimate used to optimize that actor. During execution, each vessel selects its action independently according to $a_i={\mu }_i(o_i)$, without requiring access to the observations or actions of the other agents.

Wang and Zhao [112] introduced communication-enhanced MADRL for cooperative navigation and collision avoidance, allowing vessels to exchange intent and risk information beyond their local observations. Malviya and Rajendran [39] extended multi-agent learning to both collision avoidance and path following, illustrating its broader role in cooperative navigation. Centralized critics can exploit inter-vessel information more effectively than independent learners, but scalability remains a major limitation: as fleet size increases, the joint observation–action representation expands, critic estimation becomes more difficult, and training is increasingly prone to instability [113].

Multi-agent TD3 (MATD3) addresses part of this instability by extending TD3’s twin critics, delayed actor updates, and target-policy smoothing to centralized multi-agent training. Using the smaller of two target-value estimates reduces overestimation bias, while less frequent policy updates limit oscillatory learning. Related continuous-control approaches are also being extended beyond collision avoidance. Yin and Xiang [64] jointly formulated target assignment and path planning as a distributed real-time decision problem for multiple USVs, linking mission allocation with cooperative motion planning. MATD3 can therefore provide more stable continuous-control learning than MADDPG, but its computational cost, sensitivity to hyperparameters, and limited scalability remain barriers to dense maritime traffic applications.

Multi-agent PPO (MAPPO) transfers PPO’s clipped policy updates to centralized-training–decentralized-execution settings. A centralized or shared critic evaluates the joint state during training, while each vessel executes a decentralized policy from local observations. Compared with MADDPG and MATD3, MAPPO generally offers more stable optimization and simpler implementation, making it attractive for cooperative collision avoidance, fleet navigation, and mixed-traffic decision-making [114]. Malviya and Rajendran [39] further demonstrated the applicability of PPO-based multi-agent learning to ASV collision avoidance and path following. Its performance, however, depends strongly on reward allocation, parameter sharing, and communication design. Poorly specified shared rewards can obscure individual contributions or produce coordination that improves aggregate return without guaranteeing vessel-level safety. Balancing local objectives with global traffic safety therefore remains a central challenge.

Beyond purely learned coordination, hybrid DRL offers a more deployment-oriented architecture. Its core principle is to assign interaction-intensive decisions to learned policies while retaining model-based components for global routing, rule enforcement, collision-risk assessment, and action correction. Related hierarchical reasoning has also been applied to mobile-beacon planning in uncertain ocean sensor networks, where high-level logical decisions are separated from low-level path generation to reduce decision complexity [115]. In ship navigation, such decomposition can combine the adaptability of DRL with the interpretability, feasibility, and safety guarantees of structured planning modules.

Li et al. [12] combined DQN with artificial potential fields and COLREGs-based logic, using geometric guidance and rule constraints to shape policy learning. Xu et al. [14] similarly integrated DRL with a rule-based collision-avoidance module to improve regulatory compliance. Xue et al. [15] and Wu et al. [16] embedded reciprocal velocity obstacles and the dynamic-window approach, respectively, into PPO-based planners, restricting local decisions to geometrically admissible regions. Vaaler et al. [17] added a predictive safety filter that verifies and corrects policy actions online. Together, these studies exemplify a layered architecture in which a learned policy proposes an action, a model-based module assesses its safety, and a constraint mechanism modifies unsafe outputs.

The main value of hybrid DRL lies in compensating for the limited safety guarantees, interpretability, and generalization of purely learned policies. Reward design alone cannot ensure compliance with COLREGs, vessel dynamics, or minimum-separation requirements. Rule modules, geometric planners, model predictive control, and safety filters can impose explicit boundaries while preserving the adaptive capabilities of DRL. The objective of future multi-vessel planning should therefore extend beyond maximizing cumulative reward to coordinating vessel-level safety, fleet-level efficiency, regulatory compliance, and execution feasibility within a verifiable decision architecture.

Table 7 compares representative multi-agent and hybrid DRL methods in terms of interaction setting, learning framework, coordination or safety mechanism, and principal contribution.

5. Discussion and Outlook

Ship path planning is evolving from isolated route generation into a system-level decision problem that couples dynamic traffic, navigational rules, vessel dynamics, uncertainty, and deployment constraints. Model-based methods remain essential for global guidance, risk estimation, and explicit safety enforcement, whereas DRL provides greater flexibility in high-dimensional, continuous, and interactive environments. The practical value of a planner, however, depends less on the choice of learning algorithm alone than on how observations, objectives, prior knowledge, uncertainty, and safety mechanisms are assembled into an integrated architecture.

5.1. Cross-Cutting Design Lessons

Three broad trends emerge from the reviewed literature. First, planning objectives have expanded from route length and collision avoidance to multi-objective formulations that include safety margins, COLREGs compliance, manoeuvring smoothness, energy use, and mission efficiency. Second, state representations have progressed from compact relative-motion variables to spatial risk maps, temporal histories, and multisource observations from AIS, radar, LiDAR, and electronic navigational charts. Third, hybridization has become increasingly common: search algorithms, artificial potential fields, velocity obstacles, dynamic windows, model predictive control, and safety filters are now embedded within learning systems as route priors, risk features, action constraints, or supervisory modules [12,15,16,17].

Table 8 summarizes these cross-cutting design dimensions, their roles within autonomous-navigation systems, and the limitations that remain unresolved.

Table 8 shows that planning performance emerges from the interaction of objectives, observations, prior knowledge, learning mechanisms, and validation conditions. The learning algorithm is only one element of this system. In many cases, state representation, reward formulation, rule formalization, and safety enforcement have a greater effect on practical reliability than replacing one DRL algorithm with another. Algorithm-only comparisons are therefore insufficient; future studies should report module-level ablations and evaluate complete planning architectures under standardized vessel models, traffic scenarios, disturbances, and safety criteria.

5.2. Challenges for Real-World Deployment

Despite rapid progress, real-world deployment remains constrained less by nominal planning success than by the ability to justify decisions under uncertainty. The main barriers are the formalization of navigational rules, strategic interaction with other vessels, distribution shift between training and operation, reproducible evaluation, and end-to-end safety assurance.

COLREGs contain qualitative terms such as “early,” “substantial,” and “readily apparent,” whose interpretation depends on traffic geometry, vessel dynamics, and professional judgement. Fixed thresholds or isolated reward terms cannot fully capture this context dependence. Moreover, encounter roles may change as vessels manoeuvre, particularly in multi-vessel traffic. Planning must therefore reason not only about obstacle geometry, but also about evolving responsibilities, uncertain intentions, and coupled responses.

A second limitation is the narrow distribution represented during training and evaluation. Many DRL policies assume ideal observations, fixed vessel dynamics, and a small set of scripted encounters. Sensor errors, occlusion, abnormal vessel behaviour, environmental disturbances, and unfamiliar waterways can therefore cause substantial performance degradation. Comparisons are further weakened by inconsistent vessel models, scenario definitions, risk thresholds, training budgets, and performance metrics.

Finally, the available evidence remains dominated by simulation. Real AIS data, hardware-in-the-loop experiments, and full-scale vessel trials are still limited. Purely learned policies rarely provide explicit safety guarantees, whereas hybrid architectures introduce interfaces through which perception, prediction, planning, control, and safety-filtering errors may propagate. Deployment therefore requires an end-to-end assurance process linking data quality, scenario coverage, formal constraints, system integration, staged testing, and operational certification.

Table 9 summarizes the principal deployment barriers, associated research priorities, and the evidence required to assess progress.

These challenges are tightly coupled. Improved rule formalization is of limited value if perception is unreliable; robust policies remain insufficient without explicit safety boundaries; and strong simulation results provide weak deployment evidence without reproducible benchmarks and real-system validation. Future evaluations should therefore move beyond mean cumulative return and report regulatory compliance, worst-case safety margins, constraint violations, robustness under distribution shift, computational latency, reproducibility, and the maturity of the supporting assurance process.

5.3. Priority Directions for Future Research

Three directions should receive priority.

First, safety assurance should be decoupled from reward design. Collision penalties, separation rewards, and COLREGs-related terms can shape policy preferences, but they cannot guarantee constraint satisfaction. A more credible architecture is to let the learned policy propose candidate actions and use model predictive control, control barrier functions, or predictive safety filters to verify and, when necessary, modify them. COLREGs responsibilities, ship domains, navigational boundaries, and vessel-dynamic limits can then be represented as explicit constraints rather than soft preferences [17,118,121]. Future work should also quantify intervention frequency, constraint violations, and recovery performance rather than report only average return.

Second, generalization must be evaluated through standardized, data-rich benchmarks. Test environments should combine real electronic navigational charts, AIS trajectories, heterogeneous vessel dynamics, sensor imperfections, and wind–wave–current disturbances within reproducible scenario suites. Evaluation should measure how safety and efficiency degrade across unseen vessels, waterways, traffic densities, and environmental conditions. Digital twins, domain randomization, transfer learning, meta-learning, and world models provide useful tools, but their value should be judged by performance under distribution shift and by validation using operational data [31,54,107,110].

Third, multi-vessel planning should move from reactive obstacle avoidance towards strategic interaction. Other vessels should be modelled as decision-making traffic participants with uncertain intentions, changing responsibilities, and heterogeneous manoeuvring strategies. Multi-agent reinforcement learning, game-theoretic reasoning, intention prediction, and selective communication can support coordinated decisions, while demonstrations, inverse reinforcement learning, and preference learning may improve consistency with professional maritime practice. Large language models are better suited to mission interpretation, rule explanation, and supervisory interaction than to directly producing unverified rudder or propulsion commands.

The next advances are therefore unlikely to come from adding further algorithmic variants alone. Progress will depend on integrated planning systems that combine representative data, interactive reasoning, explicit constraints, runtime assurance, and staged real-world validation.

6. Conclusion

This paper reviews the development of ship path planning from conventional search, sampling, geometric, and optimization methods to deep reinforcement learning and hybrid decision-making frameworks. Existing studies show that DRL improves the handling of high-dimensional observations, continuous control, and multi-vessel interactions, while model-based methods remain essential for route guidance, rule integration, dynamic feasibility, and explicit safety enforcement. However, real-world deployment is still limited by ambiguous COLREGs semantics, insufficient generalization, inconsistent evaluation settings, and the absence of verifiable safety guarantees. Future research should therefore focus less on developing additional algorithmic variants and more on integrating learning-based policies with explicit constraints, standardized benchmarks, digital twins, scalable multi-agent coordination, and staged validation. Such integrated and auditable architectures are necessary to bridge the gap between simulation performance and reliable autonomous navigation in real maritime environments.