Multi-Robot Collaborative Path Planning and Control Optimization Based on Deep Reinforcement Learning

1. Introduction

Multi-robot systems have gained increasing attention in logistics, inspection, and warehouse automation, where multiple robots must cooperate efficiently to accomplish complex tasks [1]. In such environments, path planning and motion control are particularly challenging under dynamic and partially observable conditions where communication is often constrained [2]. Traditional algorithms such as A*, particle swarm optimization (PSO), and genetic algorithms (GA) achieve acceptable performance in static or single-agent scenarios but typically degrade under multi-agent, real-time constraints [3]. In recent years, deep reinforcement learning (DRL) has emerged as a promising approach for autonomous decision-making in complex and uncertain environments [4]. DRL enables robots to learn navigation and coordination policies directly from interactions with the environment without relying on a global map. However, existing DRL-based models still face major limitations, including weak scalability, high computational cost during training, and insufficient integration between learned policies and motion control systems for real-world deployment [5]. Furthermore, most coordination frameworks assume full communication or perfect sensing, conditions rarely satisfied in practical applications [6]. Current methods also lack efficient real-time conflict-resolution mechanisms and often fail to maintain consistent control precision in multi-robot deployments [7].

To overcome these challenges, this study proposes a cooperative path-planning and control framework based on the deep deterministic policy gradient (DDPG) algorithm integrated with a local obstacle-avoidance module. The framework is designed for grid-based dynamic environments where robots operate under partial observability. It incorporates multi-robot cooperation to minimize conflicts while maintaining energy-efficient trajectories and stable motion control via nonlinear model predictive control (NMPC). Simulation experiments involving five to ten mobile robots in a 20×20 environment demonstrate that the proposed method reduces convergence steps by 37%, lowers the path-conflict rate to 2.3%, and improves task completion time by 21% compared with existing approaches. The findings establish deep-reinforcement-learning-based cooperative planning as an effective paradigm for decentralized multi-robot coordination, providing both theoretical insight and practical solutions for robust, real-time robotic systems in dynamic environments.

2. Materials and Methods

2.1. Sample Description and Study Environment

This study uses a 20×20 grid-based environment where 5 to 10 mobile robots operate as independent agents. Each robot is modeled as a circle with a 0.3-unit radius and can move freely in all directions. Obstacles are placed randomly, covering 10% to 30% of the area. The environment simulates limited sensing and occasional communication loss to reflect real-world challenges. Robots start from separate positions and are assigned unique destinations to avoid initial conflicts.

2.2. Experimental Setup and Control Groups

Two groups were tested: one using the proposed deep reinforcement learning method and another using a hybrid A*–PSO approach. Both groups performed the same navigation tasks in 200 trials, with identical obstacle layouts and starting conditions. The DRL group applied the Deep Deterministic Policy Gradient (DDPG) algorithm with shared parameters among agents. The comparison group used traditional path planning and a basic control system. All experiments were run on the same Python-TensorFlow simulation platform to ensure consistent results.

2.3. Measurement Procedure and Quality Assurance

We tracked four main indicators: convergence steps, collision rate, tracking error, and task duration. Each value was recorded during runtime using a central logger. To ensure reliability, each setup was repeated under 10 different random seeds. The path-following accuracy was evaluated using root mean square error (RMSE) between the planned and actual positions. Episodes with failed completion or extreme values beyond three standard deviations were excluded from analysis.

2.4. Data Processing and Model Equations

Collected data were analyzed with summary statistics and fitted curves. The average convergence steps ${\mathrm{S}}_{\mathrm{avg}}$ were calculated using [8]:

$${\mathrm{S}}_{\mathrm{avg}}={\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{s}}_{\mathrm{i}}$$

where ${\mathrm{s}}_{\mathrm{i}}$ is the step count for robot $\mathrm{i}$, and $\mathrm{N}$ is the number of robots.

The total episode reward ${\mathrm{R}}_{\mathrm{total}}$ was defined as [9]:

$${\mathrm{R}}_{\mathrm{total}}=\sum _{\mathrm{t}=1}^{\mathrm{T}}\sum _{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{r}}_{\mathrm{i},\mathrm{t}}$$

where ${\mathrm{r}}_{\mathrm{i},\mathrm{t}}$ is the reward for robot $\mathrm{i}$ at time step $\mathrm{t}$, and $\mathrm{T}$ is the total number of steps.

Statistical significance was tested using the Student’s t-test with a 0.05 threshold. Charts were generated with Matplotlib and Seaborn libraries.

3. Results and Discussion

3.1. Convergence Speed and Efficiency

In our experiments with 5–10 robots operating within a 20×20 grid environment, the proposed DRL-based cooperative planner achieved convergence in approximately 178 steps on average, representing a 37% reduction compared to the baseline A*–PSO hybrid method. This result compares favorably with findings in other multi-robot DRL studies, which reported reductions of around 25–30% under simpler settings [10,11]. The improved convergence speed is attributed to the tailored reward function design and the shared policy architecture among agents. Figure 1 shows the convergence step distributions across methods.

Figure 1. Comparison of average convergence steps between the DRL-based method and baseline algorithms under different team sizes.

Figure 1. Comparison of average convergence steps between the DRL-based method and baseline algorithms under different team sizes.

3.2. Path Conflict Rate and Task Completion Time

The cooperative control framework reduced robot-to-robot path conflict events to 2.3% in our simulations, a significant improvement over the 5–7% typical in conventional multi-robot planning frameworks [12]. Furthermore, average mission completion time improved by approximately 21%, indicating that fewer conflicts and more efficient trajectories translate directly into better overall performance. These gains demonstrate the value of integrating deep learning planning with local collision avoidance and real-time coordination [13,14].

3.3. Trajectory Tracking and Control Robustness

The motion control layer, implemented via nonlinear model predictive control (NMPC), decreased trajectory tracking error by 13% compared to standard PID controllers. Under simulated dynamic disturbances, this reduction in error contributed to stable coordination and reliable task execution. Prior work often focuses solely on planning, without deeply integrating advanced control techniques [15]. Our combined planning-control approach shows that linking DRL-based planning with robust control yields measurable benefits in both path quality and execution accuracy. Figure 2 illustrates the tracking error distributions under different control schemes.

Figure 2. Trajectory tracking errors under dynamic conditions for NMPC and PID controllers in multi-robot environments.

Figure 2. Trajectory tracking errors under dynamic conditions for NMPC and PID controllers in multi-robot environments.

3.4. Comparative Implications and Study Limitations

Compared with separate planning and control frameworks, the proposed integrated DRL-NMPC approach offers clear improvements in convergence, mission time, conflict avoidance, and tracking accuracy [16,17]. Nevertheless, this study is limited by its reliance on simulation in a 20×20 grid and a modest robot count of up to 10. Real-world scenarios with heterogeneous robot platforms, variable communication delays, and unpredictable terrain may challenge system performance. Future work should include hardware experiments, larger team sizes, and extended mission durations to validate the framework’s scalability and practical utility for real-world multi-robot systems.

4. Conclusions

This study presents a method for cooperative path planning and control in multi-robot systems working in changing environments. The approach combines a multi-agent deep deterministic policy gradient (DDPG) model with nonlinear model predictive control (NMPC) to handle path conflicts and coordination without global information. Tests in a 20×20 grid with 5–10 robots showed that the method reduced convergence steps by 37%, lowered path conflicts to 2.3%, and shortened task completion time by about 21% compared with the A*–PSO algorithm. The NMPC controller kept trajectory errors small under dynamic conditions and improved motion stability. These results confirm that a DRL-based control policy can support reliable and energy-efficient cooperation among robots. The method is suitable for applications such as warehouse logistics, inspection, and emergency response. However, its current version still depends on simulation data and has high computational cost. Future work will aim to simplify the network model and test it in real environments to improve efficiency and adaptability.