ReSwitch: Risk-Aware Switch-Time Recovery Planning for High-Speed UAV Pursuit-Evasion

1. Introduction

Unmanned aerial vehicles (UAVs) are increasingly employed in autonomous inspection, surveillance, search and rescue, environmental monitoring, and security-related missions [1,2]. As the level of autonomy increases, UAVs are expected not only to navigate toward predefined goals, but also to make responsive decisions in dynamic environments containing moving agents, obstacles, and operational constraints. In these scenarios, the control objective is often no longer a single navigation objective. Instead, the UAV must simultaneously accomplish a task, react to other agents, and maintain flight safety under limited maneuverability and finite control authority. This coupling between task execution and safety becomes particularly important when the UAV operates at high speed, because the time available for detecting risk and performing corrective maneuvers is substantially reduced.

Pursuit-evasion provides a representative setting for studying such autonomous decision-making problems [3,4,5,6]. In contrast to point-to-point navigation, the target state continuously changes according to the motion of an evader, and the pursuer must adapt its trajectory online while considering environmental constraints. The resulting decision process therefore combines adversarial interaction, motion planning, and safety-critical control. These characteristics make pursuit-evasion a useful benchmark for evaluating intelligent UAV decision-making methods, particularly when the pursuer must maintain aggressive task performance without sacrificing operational safety.

In unmanned aerial vehicle pursuit-evasion scenarios, the pursuer is required to approach or intercept an evader while maintaining a safe distance from obstacles and mission boundaries [3,4,7,8,9]. This setting is more demanding than standard navigation tasks because the decision process involves both task-oriented pursuit and safety-critical avoidance. The difficulty is further amplified when the UAV operates at high speed, where inertia, limited maneuverability, and actuation delays make late avoidance decisions unreliable [6].

Reinforcement learning provides a natural framework for pursuit-evasion because it allows a policy to learn complex interaction strategies directly from repeated experience [6,10,11]. In UAV applications, reinforcement-learning-based policies can integrate target motion, relative geometry, obstacle information, and vehicle states into a unified decision process. However, a policy trained primarily for task performance may exploit aggressive trajectories that improve interception efficiency while leaving insufficient maneuvering margin near obstacles or mission boundaries. This problem motivates the development of safe reinforcement learning and external safety mechanisms that explicitly account for unsafe states during either training or execution [12].

Traditional constrained reinforcement learning (CRL) allows the policy to trade off task performance and safety constraint during optimization [13,14,15,16]. However, in high-speed UAV pursuit-evasion scenarios, satisfying an expected safety constraint does not necessarily prevent rare but severe unsafe events. A policy may still choose aggressive pursuit actions in states where recovery is difficult but the expected safety cost remains acceptable. Therefore, safety should not always be treated as a soft trade-off against pursuit performance, especially when flight risk emerges.

An alternative direction is to separate task execution from safety intervention. Instead of requiring a single policy to simultaneously optimize pursuit performance and safety, an external safety mechanism can supervise the task policy and intervene only when necessary [17,18,19,20]. Such a design is attractive for pursuit-evasion because the pursuit policy can remain specialized for interception, while a separate controller is responsible for preventing unsafe behavior. The resulting question is then not only whether the current state is risky, but also how safety intervention should be coordinated with the task policy without unnecessarily destroying pursuit progress.

Safety shielding methods introduce an external safety module to monitor the learned policy and override unsafe actions during online execution [17,18,19,20,21], but many shielding methods rely on immediate triggering or binary safety decisions, which may be conservative and interrupt pursuit behavior unnecessarily in pursuit-evasion scenarios. In particular, a state predicted to become risky within a finite horizon does not necessarily imply that recovery must begin at the current time step [22]. The system may still possess a short recoverable interval during which the pursuit policy can be continued before safety-oriented control becomes necessary. Immediate intervention ignores this temporal structure and may therefore sacrifice task performance even when a delayed intervention remains feasible.

Recovery-based approaches further reduce the coupling between task and safety objectives by introducing a policy dedicated to returning the system toward safe states [23]. This decomposition is especially suitable for high-speed pursuit-evasion because recovery behavior can be trained independently from interception behavior and activated only when the task policy approaches the boundary of recoverability. Nevertheless, once a recovery mechanism is available, another important decision arises: the recovery policy can be activated immediately, delayed until a later feasible state, or scheduled according to the expected task value after recovery. The timing of intervention therefore becomes an independent decision variable rather than merely a consequence of risk detection.

Reachability-based methods provide principled safety analysis by determining whether the system can avoid unsafe states or remain within a safe region under admissible controls [24,25,26]. Nevertheless, their direct application to high-speed UAV pursuit-evasion is computationally expensive. Moreover, pursuit-evasion introduces an additional moving-agent dependency because the future state of the pursuer depends not only on its own control actions but also on the response of the evader. This makes exhaustive online safety analysis increasingly difficult when multiple possible recovery activation times must be evaluated within a limited control period.

Taken together, existing methods mainly address safety from three perspectives: incorporating safety costs into policy optimization, overriding unsafe actions through online shielding, or explicitly characterizing safe and unsafe regions through reachability analysis [12,20]. These approaches provide important foundations for safe autonomous control, but they leave a practical gap in high-speed pursuit-evasion. Once an emerging risk has been detected and a recovery policy is available, it remains necessary to determine whether recovery should be activated immediately or whether pursuit can be safely continued for several additional steps. This decision is particularly important because an unnecessarily early switch can substantially reduce pursuit effectiveness, whereas an excessively delayed switch can eliminate the remaining recovery margin.

These observations indicate that high-speed UAV pursuit-evasion requires a safety mechanism that does not merely impose a global constraint or trigger conservative intervention, but can prioritize recovery when risk emerges while retaining pursuit capability. This paper addresses the key question of how to ensure that a high-speed UAV pursuer gives priority to safety when flight risk emerges, while preserving as much pursuit effectiveness as possible in obstacle-rich pursuit-evasion scenarios.

Rather than treating risk detection as an immediate recovery command, we address this problem by interpreting the period between emerging risk and loss of recoverability as a finite-horizon decision window, within which different pursuit-to-recovery switching times can be explicitly evaluated according to both recovery feasibility and the pursuit capability retained after recovery.

To address this issue, we propose ReSwitch that separates task-oriented pursuit from safety-oriented recovery and coordinates them during execution. Specifically, the framework evaluates finite-horizon recoverability to determine whether the pursuit policy can still be safely continued, and activates the recovery policy when safety becomes critical. By predicting opponent-conditioned hybrid trajectories corresponding to different candidate switching times, ReSwitch evaluates whether recovery can still be completed without entering unsafe states and compares the pursuit values of the resulting post-recovery states. Consequently, the safety intervention is determined not only by whether risk exists, but also by when recovery should begin so that sufficient recoverability is retained while unnecessary interruption of pursuit is avoided.

The main contributions of this paper are summarized as follows.

• We propose ReSwitch, a risk-aware recovery framework that separates pursuit and recovery and coordinates them through finite-horizon recoverability assessment.
• An opponent-conditioned switch-time planner is designed to select a feasible recovery activation time by maximizing the discounted pursuit value after recovery.
• Experiments in obstacle-rich high-speed UAV pursuit-evasion environment demonstrates that ReSwitch achieves better safety performance than baselines while maintaining competitive pursuit performance.

2. Problem Formulation

We consider a two-player pursuit-evasion Markov game in an obstacle-rich UAV environment [4]. The game involves a pursuer and an evader. At each discrete time step t, the pursuer and the evader select actions $a_{P,t}\in {\mathcal{A}}_P$ and $a_{E,t}\in {\mathcal{A}}_E$, respectively. The system evolves according to

$$s_{t+1}\sim P(·\mid s_t,a_{P,t},a_{E,t}),$$

(1)

where P denotes the state transition kernel conditioned on the known obstacle map $\mathcal{M}$. The state space is defined as $\mathcal{S}$, and the state at step t is expressed as

$$s_t=(x_t,v_t,y_t,u_t,\mathcal{M})\in \mathcal{S}.$$

(2)

Here, $x_t$ and $v_t$ denote the position and velocity of the pursuer, whereas $y_t$ and $u_t$ denote the position and velocity of the evader. The obstacle map $\mathcal{M}$ is assumed to be known during training and evaluation and is represented as

$$\mathcal{M}={\left\{o_k\right\}}_{k=1}^N,o_k=(c_k,r_k),$$

(3)

where $c_k$ and $r_k$ are the center and radius of the k-th obstacle, respectively.

The capture set $\mathcal{G}\subset \mathcal{S}$ is defined as

$$\mathcal{G}=\left\{s_t\in \mathcal{S}\mid \parallel x_t-y_t\parallel \le r_{\mathrm{cap}}\right\},$$

(4)

where $r_{\mathrm{cap}}$ is the capture radius. The unsafe set $\mathcal{C}\subset \mathcal{S}$ is defined as

$$\begin{array}{cc}\hfill \mathcal{C}=\{s_t\in \mathcal{S}\mid & \exists k\in \{1,\dots ,N\},\parallel x_t-c_k\parallel \le r_{\mathrm{uav}}+r_k+d_{\mathrm{safe}}\hfill \\ \hfill & \mathrm{or}{x}_t\notin \mathcal{X}\}.\hfill \end{array}$$

(5)

where $\mathcal{X}$ is the admissible flight region, $r_{\mathrm{uav}}$ is the UAV safety radius, and $d_{\mathrm{safe}}$ is an additional safety margin.

Let $T_{\mathcal{G}}$ and $T_{\mathcal{C}}$ denote the first hitting times of the capture set and the unsafe set, respectively:

$$T_{\mathcal{G}}=inf\{t\ge 0\mid s_t\in \mathcal{G}\},T_{\mathcal{C}}=inf\{t\ge 0\mid s_t\in \mathcal{C}\}.$$

(6)

The pursuer aims to capture the evader while maintaining safety, which corresponds to satisfying $T_{\mathcal{G}}<T_{\mathcal{C}}$. However, in high-speed UAV pursuit-evasion, directly optimizing pursuit performance under safety constraint is challenging because unsafe events are rare but critical, and the feasibility of recovery depends strongly on short-horizon dynamics near obstacles. Therefore, this paper focuses on designing an online controller that can continue task-oriented pursuit in normal states and activate safety-oriented recovery when flight risk becomes critical.

3. Methodology

This section details the proposed ReSwitch framework. The overall architecture is illustrated in Figure 1.

The core of ReSwitch is to enforce safety-prioritized intervention through risk detection. Rather than treating safety and pursuit performance as two objectives to be directly traded off, ReSwitch prioritizes recovery once flight risk emerges. The role of the switch-time planner is to determine whether the task policy can be safely continued for a limited number of steps before recovery is activated. By predicting opponent-conditioned hybrid trajectories, the planner identifies states that are risky but still recoverable under the recovery policy. Among all feasible recovery schedules within the finite horizon, ReSwitch selects the one that satisfies the safety requirements while preserving the largest discounted pursuit value after recovery. In this way, the controller prioritizes model-predicted recoverability and preserves pursuit effectiveness whenever the predicted recovery constraints are satisfied.

3.1. State-Aware Danger Detector

During the execution of a pursuit mission, the danger detector quantifies the risk associated with continuing the task policy from the current system state. Unlike an instantaneous geometric rule based only on the current distance to an obstacle, the detector accounts for the finite-horizon evolution of the pursuer–evader system. This distinction is important in high-speed UAV flight because states with similar obstacle clearance may have substantially different safety implications depending on the pursuer velocity, evader motion, relative geometry, and the direction in which the task policy drives the system.

For any state $s_t$, the danger detector $D_H\left(s_t\right)$ is defined as the probability that the system will breach the unsafe set within a finite preview horizon H, assuming the pursuer continues with policy ${\pi }_T$ and the evader follows ${\pi }_E$:

$$D_H\left(s_t\right)={\mathbb{P}}_{{\pi }_T,{\pi }_E}\left(\exists k\in \{1,\dots ,H\},s_{t+k}\in \mathcal{C}\mid s_t\right).$$

(7)

By denoting $T_{\mathcal{C}}^{{\pi }_T,{\pi }_E}\left(s_t\right)$ as the first hitting time of the unsafe set under the joint policy execution, the danger condition can be expressed as:

$$D_H\left(s_t\right)={\mathbb{P}}_{{\pi }_T,{\pi }_E}\left(T_{\mathcal{C}}^{{\pi }_T,{\pi }_E}\left(s_t\right)\le H\mid s_t\right).$$

The horizon H represents the look-ahead steps used for risk prediction. Consequently, $D_H\left(s_t\right)$ does not indicate only whether the UAV is currently close to an obstacle or boundary. Instead, it measures whether continuation of the task-oriented pursuit behavior is likely to drive the system into the unsafe set within the preview horizon. This formulation is particularly suitable for high-speed pursuit because the current state may still be geometrically safe while its velocity and relative motion make a future violation difficult to avoid.

The true evader policy ${\pi }_E$ is unavailable to the pursuer during online execution. Therefore, an opponent model ${\widehat{\pi }}_E$ is used to approximate the evader response, yielding the practical estimate ${\widehat{D}}_{\theta }^H\left(s_t\right)\approx D_H\left(s_t\right)$ [27]. The opponent-conditioned formulation allows the detector to capture changes in future risk induced by the interaction between the pursuer and the evader rather than treating the evader trajectory as fixed.

${\widehat{D}}_{\theta }^H$ is trained using rollout-supervised binary labels. Specifically, trajectories are collected by executing the task policy ${\pi }_T$ against evader behaviors sampled from the opponent model or from a diverse set of evader strategies. For each state $s_t$ along a trajectory, the finite-horizon danger label is defined as

$$y_t^H=\mathbf{1}\left[\exists k\in \{1,\dots ,H\},s_{t+k}\in \mathcal{C}\right],$$

(8)

where $y_t^H=1$ indicates that the system reaches the unsafe set within the next H steps, and $y_t^H=0$ otherwise.

The danger detector is then trained by minimizing the binary cross-entropy loss:

$$\underset{\theta }{min}{\mathcal{L}}_D\left(\theta \right)=-{\mathbb{E}}_{s_t}\left[y_t^{H}log{\widehat{D}}_{\theta }^H\left(s_t\right)+(1-y_t^H)log\left(1-{\widehat{D}}_{\theta }^H\left(s_t\right)\right)\right].$$

(9)

After training, ${\widehat{D}}_{\theta }^H\left(s_t\right)$ provides a finite-horizon estimate of the risk associated with continuing ${\pi }_T$ from state $s_t$. A low value indicates that continuation of the pursuit policy is not predicted to reach the unsafe set within the preview horizon, whereas a high value indicates that the current pursuit behavior is approaching a region in which safety-oriented intervention may become necessary.

The detector is used with two thresholds, ${\delta }_{\mathrm{enter}}$ and ${\delta }_{\mathrm{exit}}$. The switch-time planner is activated when the danger estimate reaches the entry threshold, whereas the controller returns to normal pursuit after the danger estimate falls below the exit threshold. Setting ${\delta }_{\mathrm{exit}}<{\delta }_{\mathrm{enter}}$ creates a hysteresis region between normal pursuit and safety intervention. This prevents repeated switching caused by small variations of the detector output near a single threshold and allows recovery to continue until the state has returned to a sufficiently low-risk region.

3.2. Switch-Time Planner

When the danger detector identifies a high-risk state, ReSwitch formulates safety intervention as a finite-horizon switch-time planning problem. The core idea is to determine how long the pursuer can continue executing the task policy ${\pi }_T$ before switching to the recovery policy ${\pi }_R$. Instead of interpreting a high danger score as an unconditional command for immediate recovery, the planner evaluates whether a short period of continued pursuit remains compatible with subsequent recovery.

There is an inherent trade-off associated with the switching time. Activating ${\pi }_R$ at an early time generally preserves a larger recovery margin because the UAV is farther from the unsafe set. However, early recovery may move the pursuer away from the evader and discard useful pursuit progress. Conversely, a later switch allows ${\pi }_T$ to preserve more task-oriented behavior, but it also reduces the remaining time and maneuvering space available to the recovery policy. The planner therefore searches for a candidate schedule that retains pursuit effectiveness while satisfying the predicted recovery requirements.

Let $V_T\left(s\right)$ be the pursuit value function associated with ${\pi }_T$, and ${\pi }_R$ be the recovery policy specialized for obstacle avoidance. At time t, if ${\widehat{D}}_{\theta }^H\left(s_t\right)\ge {\delta }_{\mathrm{enter}}$, the switch-time planner is triggered. The planner enumerates candidate switching times $j\in \{0,1,\dots ,H-1\}$, where j denotes the number of steps for which the pursuer follows ${\pi }_T$ before the candidate schedule begins executing ${\pi }_R$. A smaller j therefore represents earlier recovery, whereas a larger j preserves the pursuit policy for a longer part of the predicted trajectory.

For each j, the hybrid behavioral profile is defined as:

$${\mu }_j\left(s_{t+i}\right)=\left\{\begin{array}{cc}{\pi }_T\left(s_{t+i}\right),\hfill & 0\le i<j,\hfill \\ {\pi }_R\left(s_{t+i}\right),\hfill & i\ge j.\hfill \end{array}\right.$$

(10)

Using the opponent model ${\widehat{\pi }}_E$, the planner simulates candidate trajectories $\left\{s_{t+i}^{\left(j\right)}\right\}$ [27]:

$$\begin{array}{cc}\hfill a_{P,t+i}^{\left(j\right)}& ={\mu }_j\left(s_{t+i}^{\left(j\right)}\right),\hfill \\ \hfill a_{E,t+i}^{\left(j\right)}& \sim {\widehat{\pi }}_E\left(s_{t+i}^{\left(j\right)}\right),\hfill \\ \hfill s_{t+i+1}^{\left(j\right)}& =P(s_{t+i}^{\left(j\right)},a_{P,t+i}^{\left(j\right)},a_{E,t+i}^{\left(j\right)}),s_t^{\left(j\right)}=s_t,\hfill \end{array}$$

where $P(·)$ is the state transition function. The opponent model ${\widehat{\pi }}_E$ provides the evader’s reactive actions conditioned on the predicted joint state, allowing each candidate schedule to account for the coupling between pursuit behavior, recovery behavior, and evader motion.

After switching to ${\pi }_R$ in a candidate trajectory, the planner continues the rollout until the predicted danger falls below the exit threshold or the maximum allowed recovery duration is exceeded. The recovery completion time ${\kappa }_j$ is defined as the first moment at which the predicted risk reaches the low-risk region:

$${\kappa }_j=min\left\{k\in \{j,j+1,...,j+K_{max}\}\mid {\widehat{D}}_{\theta }^H\left(s_{t+k}^{\left(j\right)}\right)\le {\delta }_{\mathrm{exit}}\right\}.$$

(11)

The quantity ${\kappa }_j-j$ therefore represents the predicted duration required by the recovery policy after the candidate switch. A candidate is considered feasible only if recovery is completed within $K_{max}$ steps, no predicted state before recovery completion belongs to the unsafe set, and the terminal state satisfies the low-risk condition. These requirements distinguish merely delaying intervention from delaying it while maintaining predicted recoverability.

A hysteresis mechanism is formed by setting ${\delta }_{\mathrm{exit}}<{\delta }_{\mathrm{enter}}$ to prevent control chattering. The optimal switch time $j^{*}$ is then determined by solving:

$$\begin{array}{cc}\hfill j^{*}=arg\underset{j\in \{0,\dots ,H-1\}}{max}& {\gamma }^{{\kappa }_j}{V}_T\left(s_{t+{\kappa }_j}^{\left(j\right)}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\kappa }_j-j\le K_{max},\hfill \\ \hfill & s_{t+i}^{\left(j\right)}\notin \mathcal{C},\forall i=1,\dots ,{\kappa }_j,\hfill \\ \hfill & {\widehat{D}}_{\theta }^H\left(s_{t+{\kappa }_j}^{\left(j\right)}\right)\le {\delta }_{\mathrm{exit}}.\hfill \end{array}$$

(12)

The objective evaluates the pursuit capability retained after the predicted recovery process. Specifically, $V_T\left(s_{t+{\kappa }_j}^{\left(j\right)}\right)$ measures the expected pursuit value of the state from which normal task-oriented behavior can be resumed, while the factor ${\gamma }^{{\kappa }_j}$ penalizes schedules requiring a longer period before returning to effective pursuit. Therefore, the planner does not simply choose the earliest feasible recovery or the latest feasible recovery. It selects the feasible schedule that provides the highest discounted post-recovery pursuit value.

The constraints in Equation (12) play complementary roles. The first constraint prevents the recovery phase from exceeding the prescribed maximum duration. The second requires the complete predicted trajectory before recovery completion to remain outside the unsafe set. The third requires the terminal state to return to the low-risk region defined by ${\delta }_{\mathrm{exit}}$. Accordingly, feasibility in ReSwitch refers to model-predicted recoverability under the learned danger detector, the opponent-conditioned rollout, and the recovery policy.

If no feasible j exists, the controller defaults to immediate recovery, $a_{P,t}={\pi }_R\left(s_t\right)$. This fallback prioritizes the safety-oriented policy when none of the predicted delayed-switch schedules satisfies the recovery constraints. If at least one feasible candidate exists, $j^{*}$ is selected according to Equation (12).

The resulting switching decision is executed in a receding-horizon manner [28]. The value $j^{*}$ represents the preferred pursuit-to-recovery schedule predicted from the current state and is not treated as a committed open-loop countdown. Only the first control action of the selected candidate schedule is executed before the state is observed again and the planning problem is updated. Consequently, if $j^{*}=0$, the selected schedule begins with recovery and the current action is generated by ${\pi }_R$. If $j^{*}>0$, the selected schedule permits continuation of pursuit at the current step, and the current action is generated by ${\pi }_T$. At the next time step, all candidate schedules are recomputed using the newly observed state.

Although all positive values of $j^{*}$ produce a pursuit action at the current step, they correspond to different predicted hybrid trajectories, different recovery completion states, and different values in Equation (12). Their role is therefore to evaluate whether continued pursuit remains preferable under the current finite-horizon prediction. Replanning after every executed action allows the switching decision to adapt to changes in the evader response and system state rather than relying on a previously predicted open-loop trajectory. Recovery is activated once the updated optimization selects $j^{*}=0$, or immediately when no feasible candidate schedule remains.

The two-threshold mechanism also determines how planning terminates. Once recovery or repeated switch-time planning drives the danger estimate below ${\delta }_{\mathrm{exit}}$, the controller returns to the task mode and ${\pi }_T$ resumes normal pursuit. If the danger estimate remains within the intervention region, the planner continues to reassess the available pursuit and recovery schedules. This closed-loop process coordinates pursuit and recovery according to the continuously updated finite-horizon risk.

During online deployment, all neural network weights are frozen. The computational workload therefore consists only of forward policy inference, opponent-model inference, danger evaluation, value evaluation, and short-horizon state propagation. At each planning step, at most H candidate schedules are evaluated, and each schedule contains at most $H+K_{max}$ predicted steps. Hence, the number of rollout steps is bounded by an order of $H(H+K_{max})$. No gradient computation or policy update is required during deployment. Moreover, the switch-time planner is bypassed when the system remains in a low-risk task state, so the additional rollout computation is concentrated around states in which safety intervention is potentially required.

The complete online execution procedure is summarized in Algorithm 1.

Algorithm 1:ReSwitch: Risk-Aware Switch-Time Recovery Planning

Require:  State $s_t$; policies ${\pi }_T,{\pi }_R$; opponent model ${\widehat{\pi }}_E$; value $V_T$; detector ${\widehat{D}}_{\theta }^H$; thresholds ${\delta }_{\mathrm{enter}},{\delta }_{\mathrm{exit}}$; horizons $H,K_{max}$; discount factor $\gamma$ 1: Initialize $m_0\leftarrow \mathrm{Task}$, where $m_t\in \{\mathrm{Task},\mathrm{Planning}\}$ 2: for$t=0,\dots ,T_{max}-1$do 3:     if $m_t=\mathrm{Task}\wedge {\widehat{D}}_{\theta }^H\left(s_t\right)<{\delta }_{\mathrm{enter}}$ then 4:         $a_{P,t}\leftarrow {\pi }_T\left(s_t\right)$, $m_{t+1}\leftarrow \mathrm{Task}$; continue 5:     end if 6:     if ${\widehat{D}}_{\theta }^H\left(s_t\right)\le {\delta }_{\mathrm{exit}}$ then 7:         $a_{P,t}\leftarrow {\pi }_T\left(s_t\right)$, $m_{t+1}\leftarrow \mathrm{Task}$; continue 8:     end if 9:     $m_t\leftarrow \mathrm{Planning}$, ${\mathcal{B}}_t\leftarrow \varnothing$ 10:     for $j=0,\dots ,H-1$ do 11:         Roll out $({\mu }_j,{\widehat{\pi }}_E)$ and compute ${\kappa }_j$ 12:         if ${\kappa }_j$ exists and $s_{t+i}^{\left(j\right)}\notin \mathcal{C},\forall i=1,\dots ,{\kappa }_j$ then 13:            ${\mathcal{B}}_t\leftarrow {\mathcal{B}}_t\cup \left\{(j,{\gamma }^{{\kappa }_j}{V}_T\left(s_{t+{\kappa }_j}^{\left(j\right)}\right))\right\}$ 14:         end if 15:     end for 16:     if ${\mathcal{B}}_t=\varnothing$ then 17:         $a_{P,t}\leftarrow {\pi }_R\left(s_t\right)$, $m_{t+1}\leftarrow \mathrm{Planning}$; continue 18:     end if 19:     $j^{*}\leftarrow arg{max}_{(j,q)\in {\mathcal{B}}_t}q$ 20:     $a_{P,t}\leftarrow {\pi }_R\left(s_t\right)$ if $j^{*}=0$, else${\pi }_T\left(s_t\right)$; $m_{t+1}\leftarrow \mathrm{Planning}$ 21: end for

4. Experiments

The experiments are conducted in a high-speed UAV pursuit-evasion environment with obstacle avoidance constraints. The pursuer is required to complete the adversarial task by locking onto the evader while avoiding obstacles during the pursuit process. This section describes the training setup, baseline methods, and evaluation metrics used in the experiments.

The experiments are conducted in a three-dimensional high-speed UAV pursuit-evasion simulation environment with obstacle constraints. Considering the maneuvering characteristics of high-speed aerial vehicles, the UAV is modeled as a three-dimensional point-mass system with constrained velocity, acceleration, and attitude variation. Both the pursuer and the evader follow the same kinematic model, while their control policies are independently optimized according to their respective objectives.

The UAV state is represented as

$${\xi }_t=(p_t,v_t,{\varphi }_t,{\theta }_t),$$

(13)

where $p_t={[x_t,y_t,z_t]}^T$ denotes the three-dimensional position, $v_t$ represents the velocity magnitude, ${\varphi }_t$ denotes the yaw angle, and ${\theta }_t$ denotes the pitch angle. The velocity vector is defined as

$${\dot{p}}_t=v_t\left[\begin{array}{c}cos{\theta }_{t}cos{\varphi }_t\\ cos{\theta }_{t}sin{\varphi }_t\\ sin{\theta }_t\end{array}\right].$$

(14)

The UAV adopts a discrete-time three-dimensional coordinated-turn point-mass model. The control input is defined as

$$a_t={[a_t^{\mathrm{lon}},a_t^{\mathrm{lat}},a_t^{\mathrm{ver}}]}^T,$$

(15)

where $a_t^{\mathrm{lon}}$, $a_t^{\mathrm{lat}}$, and $a_t^{\mathrm{ver}}$ denote the longitudinal, lateral, and vertical acceleration commands, respectively. The longitudinal acceleration controls the speed variation, while the lateral and vertical accelerations determine the turning and climbing maneuvers.

The speed update is given by

$$v_{t+1}=clip\left(v_t+a_t^{\mathrm{lon}}\Delta t,v_{min},v_{max}\right),$$

(16)

where $\Delta t$ is the simulation time step and $[v_{min},v_{max}]$ denotes the allowable velocity range.

The attitude dynamics are determined by the lateral and vertical maneuver commands. The yaw-rate and pitch-rate are calculated as

$${\dot{\varphi }}_t=clip\left({\displaystyle \frac{a_t^{\mathrm{lat}}}{v_t}},-{\omega }_{\varphi }^{max},{\omega }_{\varphi }^{max}\right),$$

(17)

$${\dot{\theta }}_t=clip\left({\displaystyle \frac{a_t^{\mathrm{ver}}}{v_t}},-{\omega }_{\theta }^{max},{\omega }_{\theta }^{max}\right).$$

(18)

Therefore, the attitude states evolve according to

$${\varphi }_{t+1}={\varphi }_t+{\dot{\varphi }}_t\Delta t,$$

(19)

$${\theta }_{t+1}={\theta }_t+{\dot{\theta }}_t\Delta t.$$

(20)

The position update is expressed as

$$p_{t+1}=p_t+v_{t+1}\left[\begin{array}{c}cos{\theta }_{t+1}cos{\varphi }_{t+1}\\ cos{\theta }_{t+1}sin{\varphi }_{t+1}\\ sin{\theta }_{t+1}\end{array}\right]\Delta t.$$

(21)

The action space of each UAV is therefore defined as

$$a_t={[a_t^{\mathrm{lon}},a_t^{\mathrm{lat}},a_t^{\mathrm{ver}}]}^T\in \mathcal{A},$$

(22)

where the acceleration commands satisfy

$$|a_t^{\mathrm{lon}}|\le a_{\mathrm{lon}}^{max},$$

(23)

$$|a_t^{\mathrm{lat}}|\le a_{\mathrm{lat}}^{max},\left|a_t^{\mathrm{ver}}\right|\le a_{\mathrm{ver}}^{max}.$$

(24)

The simulation environment is constructed as a bounded three-dimensional flight region containing randomly generated spherical obstacles. The obstacle map is represented as

$$\mathcal{M}={\left\{o_k\right\}}_{k=1}^N,o_k=(c_k,r_k),$$

(25)

where $c_k={[x_k,y_k,z_k]}^T$ and $r_k$ denote the center position and radius of the k-th obstacle, respectively. During each episode, the initial states of the pursuer, evader, and obstacles are randomly sampled while satisfying the minimum separation constraints. The pursuer aims to capture the evader by reducing the relative distance to the capture radius, whereas the evader attempts to avoid interception while maintaining collision-free flight.

The admissible flight region is defined as

$$\mathcal{X}=\left\{(x,y,z)\right|x\in [x_{min},x_{max}],y\in [y_{min},y_{max}],z\in [z_{min},z_{max}]\}.$$

(26)

The main parameters are summarized in Table 1. The simulation time step is set to $\Delta t$. The UAV velocity, acceleration, and attitude variation are constrained according to the physical maneuverability limits of high-speed aerial vehicles. These constraints prevent unrealistic instantaneous direction changes and require the pursuer to balance interception performance with future safety margins during obstacle-rich pursuit-evasion.

4.1. Training Setup

The training process of the proposed method consists of three parts: policy-network training, opponent-policy learning, and danger-detector training. The details of each part are described as follows.

1.

PPO-based policy training. Both the pursuit policy ${\pi }_T$ and the recovery policy ${\pi }_R$ are trained using PPO [29]. PPO optimizes a stochastic actor ${\pi }_{\eta }$ and a critic network $V_{\psi }$ by solving the following clipped policy optimization problem:

$$\underset{\eta ,\psi }{max}{\mathbb{E}}_t\left[min\left(r_t\left(\eta \right){\widehat{A}}_t,\mathrm{clip}\left(r_t\left(\eta \right),1-ϵ,1+ϵ\right){\widehat{A}}_t\right)-c_v{\left(V_{\psi }\left(s_t\right)-{\widehat{R}}_t\right)}^2+c_e\mathcal{H}\left({\pi }_{\eta }(·\mid s_t)\right)\right],$$

(27)

where

$$r_t\left(\eta \right)={\displaystyle \frac{{\pi }_{\eta }(a_t\mid s_t)}{{\pi }_{{\eta }^{\mathrm{old}}}(a_t\mid s_t)}}.$$

Here, ${\widehat{A}}_t$ is the advantage estimate, ${\widehat{R}}_t$ is the return target, $ϵ$ is the clipping coefficient, and $c_v$ and $c_e$ are the weights of the value loss and entropy regularization, respectively.

The pursuit policy ${\pi }_T$ is trained to track and intercept the evader, and the critic obtained during pursuit-policy training is retained as the pursuit value function $V_T$ and is later used by the switch-time planner. The reward function of the pursuit policy consists of three terms: distance reward, angle reward, and boundary reward. Following the reward design in [7], the pursuit reward is defined as

$$R_T={\omega }_1{R}_{\mathrm{distance}}+{\omega }_2{R}_{\mathrm{angle}}+{\omega }_3{R}_{\mathrm{boundary}},$$

(28)

where ${\omega }_1$, ${\omega }_2$, and ${\omega }_3$ are the corresponding weighting coefficients.

The distance reward measures the improvement in the pursuer–evader distance between two consecutive time steps:

$$R_{\mathrm{distance}}=\mathrm{clip}\left({\displaystyle \frac{d_{t-1}-d_t}{R_{max}}},-1,1\right),$$

(29)

where $d_t=\parallel x_t-y_t\parallel$ denotes the pursuer–evader distance at time step t, and $R_{max}$ is a normalization coefficient. A reduction in the pursuer–evader distance produces a positive reward, whereas an increase in the distance produces a negative reward.

The angle reward measures the improvement in the angular error between the pursuer’s current motion direction and the line of sight toward the evader:

$$R_{\mathrm{angle}}=\mathrm{clip}\left({\displaystyle \frac{{\theta }_{t-1}-{\theta }_t}{{\theta }_{max}}},-1,1\right),$$

(30)

where ${\theta }_t$ denotes the angular error at time step t, and ${\theta }_{max}$ is the normalization angle. A decrease in the angular error produces a positive reward, encouraging the pursuer to align its motion direction with the evader, whereas an increase in the angular error produces a negative reward.

The boundary reward penalizes the pursuer when it approaches the boundary of the admissible flight region:

$$R_{\mathrm{boundary}}=-max\left(0,1-{\displaystyle \frac{c_b}{d_{\mathrm{boundary}}}}\right),$$

(31)

where $c_b$ denotes the minimum clearance between the pursuer and the nearest flight boundary, and $d_{\mathrm{boundary}}$ determines the effective range of the boundary penalty. When $c_b\ge d_{\mathrm{boundary}}$, no boundary penalty is imposed. As the pursuer approaches the boundary, the penalty gradually increases toward $-1$.

In the experiments, the reward weights and normalization parameters are set to

$${\omega }_1=4,{\omega }_2=2,{\omega }_3=4,R_{max}=800,{\theta }_{max}={20}^{\circ },d_{\mathrm{boundary}}=200.$$

(32)

Substituting these values into Equations (28)–(31), the pursuit reward used in the experiments is

$$\begin{array}{cc}\hfill R_T=& 4\mathrm{clip}\left({\displaystyle \frac{d_{t-1}-d_t}{800}},-1,1\right)+2\mathrm{clip}\left({\displaystyle \frac{{\theta }_{t-1}-{\theta }_t}{{20}^{\circ }}},-1,1\right)\hfill \\ \hfill & -4max\left(0,1-{\displaystyle \frac{c_b}{200}}\right).\hfill \end{array}$$

(33)

In addition, a terminal reward of $+150$ is assigned for successful capture, whereas a terminal penalty of $-180$ is imposed for failure.

The recovery policy ${\pi }_R$ is trained to reduce safety risk near obstacles and mission boundaries. Compared with the pursuit reward, the distance reward and angle reward are removed. The recovery reward is defined as

$$R_R={\omega }_4{R}_{\mathrm{boundary}}+{\omega }_5{R}_{\mathrm{obstacle}},$$

(34)

where ${\omega }_4$ and ${\omega }_5$ are weighting coefficients. In the experiments, they are set to

$${\omega }_4=3,{\omega }_5=4.$$

(35)

2.

Opponent-model training. An opponent model ${\widehat{\pi }}_E$ is learned from observed evader behavior data [27]. The evader trajectories are collected during the training of the pursuit policy and the recovery policy since the true evader policy is unavailable during online execution. Given an evader behavior dataset ${\mathcal{D}}_E={\left\{(s_i,a_{E,i})\right\}}_{i=1}^{N_E}$, the opponent model is trained by behavior cloning [30]:

$$\underset{\omega }{min}{\mathcal{L}}_{\mathrm{BC}}\left(\omega \right)=-{\mathbb{E}}_{(s,a_E)\sim {\mathcal{D}}_E}[log{\widehat{\pi }}_{E,\omega }(a_E\mid s)].$$

(36)

After training, ${\widehat{\pi }}_{E,\omega }$ is denoted as ${\widehat{\pi }}_E$ and is used to predict evader responses in opponent-conditioned hybrid rollouts.

3.

Danger-detector training. The danger detector ${\widehat{D}}_{\theta }^H$ is trained using rollout-generated binary labels. A state is labeled as dangerous if continuing the task policy ${\pi }_T$ leads the UAV to enter the unsafe set within the finite horizon H. The binary danger labels are generated according to Equation (8), and the detector is trained by minimizing the loss in Equation (9).

The overall training procedure is organized into three stages:

1.

Pursuit-policy training. The pursuit policy ${\pi }_T$ is trained with the pursuit reward in Equation (28) using a curriculum learning strategy [31], in which the initial pursuer–evader distance is gradually increased and the number of obstacles is progressively increased to nine.

2.

Recovery-policy training. The recovery policy ${\pi }_R$ is trained with the recovery reward in Equation (34) until the constraint violation rate converges to a low level.

3.

Opponent-model and danger-detector training. The opponent model ${\widehat{\pi }}_E$ is trained from observed evader trajectories using behavior cloning. The danger detector ${\widehat{D}}_{\theta }^H$ is then trained for 30,000 steps using rollout-supervised binary labels.

4.2. Evaluation Metrics

In the high-speed UAV pursuit-evasion game, the UAV has two main objectives. First, it should complete the adversarial task by successfully locking onto the evader. Second, it should maintain safe flight by avoiding obstacles during the pursuit process. Based on these objectives, four evaluation metrics are used.

• Success Rate: The success rate measures the percentage of episodes in which the pursuer successfully completes the adversarial task:

  $$R_{\mathrm{success}}={\displaystyle \frac{N_{\mathrm{episode}}^{\mathrm{success}}}{N_{\mathrm{episode}}^{\mathrm{total}}}},$$

  (37)

  where $N_{\mathrm{episode}}^{\mathrm{success}}$ is the number of successful episodes and $N_{\mathrm{episode}}^{\mathrm{total}}$ is the total number of evaluation episodes.
• Safety Rate: The safety rate measures the percentage of episodes in which the pursuer does not collide with any obstacle or violate the flight boundary during the entire rollout:

  $$R_{\mathrm{safe}}=1-{\displaystyle \frac{N_{\mathrm{episode}}^{\mathrm{danger}}}{N_{\mathrm{episode}}^{\mathrm{total}}}}.$$

  (38)
  Here, $N_{\mathrm{episode}}^{\mathrm{danger}}$ denotes the number of episodes in which the pursuer collides with an obstacle or violates the flight boundary during the entire rollout.
• Zero-Danger Rate: The zero-danger rate measures the percentage of episodes in which no unsafe state occurs. Here, an unsafe state refers to a near-obstacle or near-boundary state in which the pursuer’s clearance to an obstacle or the flight boundary is smaller than $r_{\mathrm{unsafe}}=5$.

  $$R_{\mathrm{zero}}={\displaystyle \frac{N_{\mathrm{episode}}^{\mathrm{safe}}}{N_{\mathrm{episode}}^{\mathrm{total}}}},$$

  (39)
• Average Capture Time: The average capture time measures the mean number of time steps required to capture the evader over successful episodes only:

  $$T_{\mathrm{cap}}={\displaystyle \frac{1}{N_{\mathrm{episode}}^{\mathrm{success}}}}{\displaystyle \sum _{i=1}^{N_{\mathrm{episode}}^{\mathrm{success}}}}{T}_{\mathcal{G}}^{\left(i\right)},$$

  (40)

  where $T_{\mathcal{G}}^{\left(i\right)}$ denotes the capture time of the i-th successful episode.

The key hyperparameters used in the environment configuration and danger-detector training are summarized in Table 1. These parameters define the state-action setting, obstacle distribution and flight constraints used in the experiments.

4.3. Comparison with Baselines

To evaluate the effectiveness and robustness of the proposed method, all methods are evaluated using 10 independent random seeds. For each random seed, all trainable components required by the corresponding method are retrained from scratch, resulting in 10 independently trained models for each method. After training, each model is evaluated over 5000 independently generated pursuit-evasion trajectories. Therefore, each method is evaluated using 50,000 trajectories in total across the 10 independently trained models.

We compare ReSwitch with the following three baseline methods.

• ReSwitch is the proposed method. It uses the finite-horizon danger detector to estimate the risk of continuing the pursuit policy and employs the switch-time planner to select a feasible recovery activation time that preserves pursuit value.
• Pursuit Only uses only the pursuit policy ${\pi }_T$ for target tracking and interception. No recovery policy, danger detector, or safety intervention mechanism is used during execution.
• Immediate Recovery uses the same danger detector and recovery policy as ReSwitch, but does not perform switch-time planning. Once the danger score satisfies

  $${\widehat{D}}_{\theta }^H\left(s_t\right)\ge {\delta }_{\mathrm{enter}},$$

  the controller immediately switches to the recovery policy ${\pi }_R$.
• PPO-Lagrangian optimizes a constrained reinforcement learning objective [13,15]:

  $$\underset{\pi }{max}{\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{t=0}^T}{\gamma }^{t}r(s_t,a_t)\right]\mathrm{s}.\mathrm{t}.{\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{t=0}^T}{\gamma }^{t}c(s_t,a_t)\right]\le d,$$

  where r is the same reward function used for training the pursuit policy, and c is defined using the same safety-related terms as the recovery reward.

For each evaluation trajectory, the initial positions of the pursuer, evader, and obstacles are randomly generated subject to the environment constraints described in Section 4.1. Representative trajectory visualizations are shown in Figure 2.

Figure 3 provides a direct visualization of the safety–pursuit trade-off. The horizontal axis represents the safety rate, and the vertical axis represents the pursuit success rate. Therefore, moving toward the right indicates improved safety performance, whereas moving upward indicates improved task completion capability. The Pursuit Only policy achieves relatively high pursuit efficiency but suffers from insufficient safety performance. In contrast, Immediate Recovery improves safety by switching to the recovery policy conservatively, but its pursuit capability is reduced due to frequent intervention. PPO-Lagrangian provides an intermediate compromise by incorporating safety constraints into policy optimization. The proposed ReSwitch method is located in the upper-right region of the trade-off space, demonstrating that the switch-time planning mechanism can preserve pursuit capability while maintaining reliable safety performance.

As shown in Table 2, the results averaged over 10 independently trained models show that ReSwitch provides a favorable balance between pursuit effectiveness and flight safety. The confidence intervals additionally characterize the sensitivity of each method to random initialization and stochasticity in the training process.

The Pursuit Only baseline achieves the shortest average capture time among successful episodes, with a mean of 48.6 steps, because its controller focuses entirely on pursuit without safety-oriented intervention. However, this aggressive behavior results in substantially lower mean safety and zero-danger rates, 61.4% and 24.8%, respectively.

PPO-Lagrangian improves the mean safety rate to 82.6% by incorporating an expected safety constraint during policy optimization. Nevertheless, its mean success rate is 65.3%, and the mean capture time increases to 119.2 steps. These results indicate that incorporating safety into a single constrained optimization objective can reduce unsafe behavior, but may also substantially affect pursuit efficiency.

Immediate Recovery obtains the highest mean safety rate and zero-danger rate, reaching 94.8% and 76.5%, respectively. However, immediately activating the recovery policy whenever the danger threshold is exceeded results in frequent interruption of task-oriented behavior. Consequently, its mean success rate decreases to 53.2%, while its mean capture time increases to 123.5 steps.

In comparison, ReSwitch achieves the highest mean success rate of 74.1%, while maintaining a mean safety rate of 92.7% and a mean zero-danger rate of 72.8%. Its mean capture time is 85.4 steps, which is substantially shorter than those of PPO-Lagrangian and Immediate Recovery. These results show that explicitly optimizing the recovery activation time can retain more pursuit capability than immediate intervention while maintaining a high level of safety.

4.4. Ablation Study

To further investigate the contribution of switch-time planning and the value-preserving selection objective, we conduct an ablation study using three variants of the proposed framework. The same evaluation protocol as in Section 4.3 is adopted. Specifically, each variant is evaluated using 10 independent random seeds, with the corresponding trainable components retrained independently for each seed. Each trained model is then evaluated using 5000 pursuit-evasion trajectories. The reported results are the mean and 95% confidence interval across the 10 random seeds.

The evaluated variants are defined as follows.

• Immediate Recovery removes switch-time planning and activates the recovery policy immediately when the danger score reaches the entry threshold. This variant represents the most conservative intervention strategy.
• Latest Recovery retains the feasibility evaluation of candidate switching times but removes the value-preserving objective. Among all feasible candidates, it always selects the latest feasible switching time:

  $$j_{\mathrm{latest}}^{*}=max\left\{j\mid j\in {\mathcal{J}}_{\mathrm{feasible}}\right\},$$

  (41)

  where ${\mathcal{J}}_{\mathrm{feasible}}$ denotes the set of switching times satisfying the recovery constraints in Equation (12). This variant evaluates whether simply delaying recovery as much as possible is sufficient.
• Random Recovery Time randomly selects one switching time from the feasible candidate set ${\mathcal{J}}_{\mathrm{feasible}}$. This variant retains recoverability checking but removes the pursuit-value-based selection criterion.
• ReSwitch uses the complete switch-time planner and selects the feasible candidate that maximizes the discounted post-recovery pursuit value according to Equation (12).

Table 3 summarizes the ablation results across the 10 independent training seeds. The results indicate that the choice of recovery activation time substantially affects the balance between pursuit effectiveness and safety.

Immediate Recovery provides the most conservative behavior, achieving the highest mean safety rate of 94.8% and the highest mean zero-danger rate of 76.5%. However, its mean success rate is only 53.2%, and the average capture time increases to 123.5 steps. This result illustrates the performance cost associated with activating the recovery policy immediately whenever risk is detected.

Latest Recovery produces a higher mean success rate of 61.7% and reduces the mean capture time to 93.1 steps, but its safety performance deteriorates. In particular, its mean safety rate and zero-danger rate decrease to 84.3% and 55.6%, respectively. This suggests that delaying recovery to the latest feasible time can excessively reduce the remaining recovery margin.

Random Recovery Time achieves intermediate results, with a mean success rate of 58.4%, a mean safety rate of 87.9%, and a mean zero-danger rate of 61.7%. Although restricting the selection to feasible switching times prevents completely unconstrained delay, random selection does not explicitly account for the quality of the post-recovery state and therefore cannot consistently preserve pursuit progress.

By contrast, ReSwitch achieves a mean success rate of 74.1%, which is the highest among the evaluated variants, while maintaining a mean safety rate of 92.7% and a mean zero-danger rate of 72.8%. Its mean capture time is 85.4 steps, lower than those of the other recovery-based variants. These results support the role of the value-preserving switch-time objective: rather than always recovering immediately or delaying recovery as much as possible, ReSwitch selects a feasible switching schedule according to the pursuit capability retained after recovery.

5. Conclusion

ReSwitch is proposed as a Risk-Aware Switch-Time Recovery Planning framework for safe decision making in high-speed UAV pursuit-evasion. This method separates task-oriented pursuit and safety-oriented recovery through a finite-horizon danger detector and a value-preserving switch-time planner. Instead of triggering recovery immediately after risk detection, ReSwitch evaluates candidate switching times and selects a feasible hybrid policy that maintains recoverability while preserving pursuit value. Evaluations in obstacle-rich pursuit-evasion scenarios demonstrate that ReSwitch achieves a favorable trade-off between adversarial task completion and safe flight, maintaining safety performance close to immediate recovery while substantially improving pursuit efficiency.

For future work, we aim to extend ReSwitch to multi-UAV pursuit-evasion scenarios. Specifically, we plan to improve opponent-policy estimation under unseen evader behaviors and optimize the rollout planner for real-time deployment in more complex dynamic environments.