AEGIS: An Age-Aware, Evidence-Guarded Safe-Action Layer Fusing RGB-D Sensing with Vision–Language Monitoring for Collision-Aware Robotic Manipulation

1. Introduction

Learned and classical manipulation policies are increasingly capable, yet deploying them safely on physical, low-cost hardware remains difficult. A policy that maximises task reward does not, by construction, guarantee that every intermediate command it emits is collision-free, inside the reachable workspace, within joint and velocity limits, and free of abrupt slews that a real actuator cannot follow. On expensive, heavily instrumented platforms these constraints are often enforced by proprietary motion-planning stacks; on low-cost manipulators such as the Niryo NED3 Pro they are not, and an unsafe intermediate command can drive the arm into an obstacle, a person, or a joint hard-stop before the task-level policy ever fails. This mismatch between task success and step-wise physical admissibility is the deployment gap addressed in this paper; it is a counterpart, at the command-safety level, to the sim-formulation-to-hardware gap documented for a low-level adaptive controller on the same NED3 Pro actuator, where friction, current limits, and communication latency weaken paper-level guarantees [1].

The gap is especially acute for two reasons that are specific to modern manipulation. First, learned policies (behaviour cloning, reinforcement learning, and action-chunking imitation) emit commands that are statistically good but occasionally out-of-distribution, and a single bad command can cause a collision even when the average behaviour is safe. Second, safety is only partially geometric: some hazards, such as a human hand entering the workspace, are semantic and require perception beyond distance thresholds. These semantic hazards are resolved by analysing the data stream of the workspace camera: rather than thresholding distances alone, a vision–language model interprets the raw RGB frames from a fixed RGB-D sensor to judge whether a detected intruder is a person that warrants a protective stop, so the semantic monitor is a sensor-data-analysis layer built on the same visual input that drives camera-based object localisation. A practical safety architecture for low-cost manipulation must therefore combine continuously enforced geometric constraints with an optional semantic monitor, and must do so without assuming access to the policy’s internals or the ability to retrain it.

This paper presents AEGIS, a safe-action projection whose hold radius is an explicit function of a semantic belief and of the age of the verdict that produced it, together with the sensing and safety architecture it runs in: a geometric constrained-Markov-decision-process (CMDP) projection layer, a reactive obstacle detour, and a vision–language-model (VLM) monitor, all driven by one fixed RGB-D camera on a Niryo NED3 Pro and validated in the MuJoCo simulator and closed-loop on the physical arm. Figure 1 gives the complete picture in one place: the sensing chain, the belief and its decay, the resulting hold radius, the projection that consumes it, and what the whole loop measures on the physical arm. The layer is policy-agnostic (it wraps a classical RRT motion planner and a learned Action-Chunking-Transformer (ACT) identically) and it acts on realisable commands rather than on the reward, so no retraining is required to apply the stated grasp-site and command constraints to an existing policy.

As with hardware-oriented control studies on the same low-cost platform [1], most of the individual mechanisms used here (constraint projection, damped-least-squares deflection, tangent–arc shortest-path detours, action chunking, and self-imitation) are established techniques and are not claimed as new in isolation. The specific mechanism studied here is the age-aware, evidence-guarded intervention for safety (AEGIS) of Section 4, whose enforced radius depends on the semantic score and the age of the corresponding verdict. Related delay- and age-aware methods are discussed in Section 2; the contribution claimed here is the formulation and integration used for this manipulation architecture. The remaining contribution is the integrated, reproducible safety architecture and evaluation methodology: a policy-agnostic geometric admissibility layer coupled to a semantic monitor, to redundant RGB-D sensing, and to a simulation-as-pre-screen analysis, evaluated on a low-cost manipulator across static, dynamic, and adversarial obstacle regimes (in simulation for the policy comparison and closed-loop on the physical arm for the safety and sensing chain) with explicit reporting of where the guarantees hold and where task completion saturates.

The main contributions of the paper are as follows:

• A safety margin that is an explicit function of the age of its own evidence: AEGIS makes the hold radius depend on a semantic belief and on the age of the corresponding verdict, with a conditional grasp-site separation result and a fresh-object path-feasibility property, together with a buffer-sizing condition stated in terms of a lower bound on the semantic score. The distance decomposition is informed by speed-and-separation concepts, but the prototype is not an implementation or certification of ISO/TS 15066 [2].
• A sensing chain evaluated against a measured latency budget: A single fixed RGB-D camera supplies planar object localisation, workspace-gated intruder detection, metric depth as a complementary confirmation channel, and an on-device vision–language verdict. The measured reaction times support keeping geometric checks active for moving intrusions and using the semantic monitor to modulate the response to sufficiently persistent scene conditions; they are reported as observed timings rather than certified interception bounds.
• Closed-loop evaluation on a low-cost physical manipulator: The integrated sensing and control chain routes around an inert obstacle and holds for a detected human hand. In the reported physical trials, the AEGIS runs retained positive internally measured clearance, one trust-latest run ended up inside the detected footprint after a detection dropout, and the worst-case policy showed reduced task availability. The common-buffer comparison that isolates the age-indexed rule is reported in Table 6.
• A policy-agnostic geometric layer that requires no retraining: The layer checks proposed commands against eight geometric constraints, four of them forward-kinematics-coupled and holds when no checked command is admissible. It wraps a classical RRT planner and a learned action-chunking transformer without retraining; that comparison is run in simulation and reported in the Supplementary Material. In the reported simulation audit, it modified only a small fraction of commands and eliminated the reported grasp-site obstacle contacts in the tested exploration settings.

2. Related Work

Safe control of learned and classical manipulation policies draws on three lines of work: constrained decision-making and safe reinforcement learning, run-time safety enforcement (shielding and action projection), and imitation learning for manipulation, together with the broader question of transferring safety guarantees from simulation to hardware.

Constrained Markov decision processes provide the standard formalism for balancing reward and safety cost, and the modern safe reinforcement-learning literature (surveyed comprehensively by Gu et al. [3]) optimises policies to respect expected-cost constraints through constrained-optimisation, Lagrangian, and safety-layer methods, supported by standardised benchmarks and tooling such as Safety Gymnasium and OmniSafe [4,5]. These methods shape behaviour during training but provide expectation-level rather than per-step guarantees, and they require access to the training loop. In contrast, the present work enforces a hard, per-step geometric constraint at run time through projection, so it applies to already-trained or non-learned policies without modifying them.

Run-time enforcement has been studied as shielding, where a correct-by-construction shield overrides unsafe actions [6], and as control-barrier-function and safety-filter methods that minimally modify a nominal command to keep the state in a safe set [7,8,9]. Action projection (returning the closest admissible action) is the mechanism used here; it is closely related to safety filters but expressed directly in the manipulator’s command and forward-kinematics geometry, which keeps it light enough to run at control rate on a low-cost platform. These formulations do admit imperfect information: robust and stochastic barrier formulations inflate the safe set to cover bounded state-estimation error, and predictive filters propagate that uncertainty over a horizon [9]. What they model, though, is uncertainty in a continuously available state signal. The uncertainty that dominates a semantic monitor is different in kind: the classification is discrete, refreshed only every $\Delta$ seconds, and its reliability decays with the age of the last verdict rather than with sensor noise. A margin that is correct the instant a verdict is emitted can be badly wrong a few hundred milliseconds later, with no state estimate having become noisier. Section 4 addresses this issue in the present manipulation setting. Delay- and age-aware safety have also been studied in related settings. Ballotta and Talak [10] use age-of-information concepts for barrier-based multi-robot control under communication delays. Chen and Chandra [11] combine asynchronous VLM regulation with an explicit staleness indicator and a conservative fallback for visual navigation. The present work differs in its clearance interpolation, RGB-D manipulation pipeline, and physical low-cost manipulator evaluation.

Physical human–robot coexistence has its own safety literature, organised around the collaborative modes of ISO/TS 15066 [2]: speed-and-separation monitoring maintains a distance sized by the robot’s reaction and the human’s approach [12,13], whereas power-and-force limiting instead detects and reacts to contact once it has occurred [14]. Both presume a continuously available sensing channel with a certified response time (a safety-rated scanner, a light curtain, joint-torque sensing) and both are deliberately class-agnostic, treating every intrusion identically. Vision supplies the class that these modes discard, but it supplies it on a different timescale: a learned pose detector such as OpenPose [15] reports per-frame inference times in the tens of milliseconds, against the millisecond-order response a safety-rated scanner is certified for, which reintroduces precisely the staleness this paper is about. Surveys of safe interaction [16] organise these modes without treating the age of a semantic judgement as a design variable; Section 4 does.

For the task policies, sampling-based motion planning (see [17] for a recent comparative review) supplies the classical baseline, while imitation learning supplies the learned baseline. Action-chunking transformers [18] predict short action chunks and combine them by temporal ensembling, which has proven effective for fine manipulation on low-cost hardware; when the demonstrator is imperfect, dataset-aggregation (DAgger) and self-imitation (surveyed in [19]) can improve coverage. The present study uses ACT with temporal ensembling as the learned policy and applies self-imitation to push completion on the hardest regime. Semantic hazards are handled by a lightweight vision–language-model monitor; large multimodal models are increasingly used as high-level planners and as policies in robotics [20,21], and here a VLM is used instead only as an additional veto layer on top of the geometric guarantee. A recent survey of vision–language–action safety identifies the resulting safety–latency trade-off (a monitor slow enough to reason semantically may be too slow to prevent the hazard it detects) as a central open problem [22], and even edge-served language monitors report inference latencies around a quarter second [23]; Section 5.3 makes this trade-off quantitative for the present architecture using reaction times measured on the physical platform.

Finally, the value of a safety benchmark rests on whether its guarantees transfer. Sim-to-real transfer and domain randomisation are well studied for skill transfer [24], and the MuJoCo physics engine [25] is the standard testbed. Less studied is the use of simulation explicitly as a pre-deployment safety screen. The present paper contributes to this question by measuring how well per-constraint detectors evaluated in simulation predict unsafe events on a domain-randomised surrogate of the real platform, complementing task-level sim-to-real studies with a safety-level analysis.

3. Materials and System Architecture

The architecture is a single closed loop, and the subsections that follow describe its stages in the order the loop executes them, following Figure 1; its lower panels are the mechanism and the hardware outcome that Section 4 and Section 5.2.3 establish. Three properties of the arrangement matter for what follows and are visible in the figure. First, every quantity the safety layer acts on comes from one fixed RGB-D camera: the colour channel supplies the intruder’s position through the calibrated homography, and the depth channel supplies complementary confirmation rather than a second coordinate estimate. Second, the semantic monitor is off the control loop, running asynchronously and depositing a timestamped verdict, so a slow or failed query delays the verdict but never stalls the arm. Third, the two layers meet at exactly one number, the hold radius ${\rho }_{\mathrm{eff}}$; everything AEGIS contributes is contained in how that number is computed from the verdict and its age, which is why the comparison in Section 5.2.3 can isolate the mechanism by changing that rule alone.

3.1. Simulation Platform and Evidence Pipeline

Experiments were conducted in MuJoCo [25] with a model of the Niryo NED3 Pro and a parallel-jaw gripper. The action is seven-dimensional and the observation 32-dimensional, augmented to 38 for obstacle-aware learned policies. All policies run behind the same safety wrapper, and every episode logs the raw command, the projected command, the per-constraint intervention flags and the achieved state, so success and step-wise admissibility are measured independently. The same architecture then runs on the physical NED3 Pro through a real-robot adapter, with a kinematic twin driving the geometric layer.

Claims are established at three levels of physical fidelity, and every result below names the tier that produced it (Table 1). The labelling is part of the contribution: unseparated tiers invite a reader to transfer a guarantee proved for a point mass to a physical manipulator.

Figure 2. The physical rig. (a) The laboratory setup. (b) The same arrangement in schematic form: the camera is eye-to-hand (rigidly fixed, never moved once calibrated) and looks across and slightly down at the table, so the workspace lies between the camera and the arm. The calibrated region and the two carry corners A and B are marked; the intruder is introduced at the midpoint.

Figure 2. The physical rig. (a) The laboratory setup. (b) The same arrangement in schematic form: the camera is eye-to-hand (rigidly fixed, never moved once calibrated) and looks across and slightly down at the table, so the workspace lies between the camera and the arm. The calibrated region and the two carry corners A and B are marked; the intruder is introduced at the midpoint.

3.2. Task Formulation and Safety Constraints

The benchmark task is pick-and-place around an obstacle. An object is placed on one side of the workspace, an obstacle occupies the region between the object and a target, and the object must be carried to the target on the opposite side and released within a tolerance without contacting the obstacle. Task success is defined as the object coming to rest within $7\mathrm{cm}$ of the target on the table while the arm never registers an obstacle contact. The safety objective is orthogonal to task success: it requires that every commanded state be geometrically admissible. Writing the grasp-site (tool) position as $p\left(q\right)\in {\mathbb{R}}^3$ via forward kinematics from joint vector q, the admissible set $C\left(s\right)$ at state s is the intersection of eight geometric constraints (joint limits, joint velocity, command smoothness, a workspace box, a reach sphere, a table Z-floor, dynamic-obstacle clearance, and gripper-command sanity), detailed in the next subsection. The safety task and the manipulation task are thus decoupled: the projection layer enforces the implemented constraints at the commanded grasp site, while the policy is responsible for completing the manipulation within that envelope. Three workspace variants are used, with measured object-to-target distances of 26, 30 and 38 cm; the narrow front-back extent makes in-plane routing around a large obstacle difficult, which becomes the binding constraint in the varied-height regime.

3.3. Geometric Safe-Action Projection Layer

Formally, the benchmark is a constrained Markov decision process $(\mathcal{S},\mathcal{A},P,r,\left\{c_k\right\},\left\{d_k\right\})$ in which the task reward r is accompanied by per-constraint cost signals $c_k(s,a)\in \{0,1\}$, one per geometric constraint, with budgets $d_k=0$: a policy is feasible only if it never incurs a geometric violation. Rather than solving this CMDP by constrained policy optimisation (which yields expectation-level guarantees and requires access to training) the benchmark enforces feasibility at run time: a projection layer restricts every executed action to the admissible set, so that any policy composed with the layer satisfies the per-step constraints $c_k=0$ by construction, up to the discretisation caveats stated below. This is the shield-style reading of the CMDP: the projection converts a constraint on expected cumulative cost into a per-step admissibility condition that holds for arbitrary, even adversarial, task policies.

Concretely, the layer is placed between the policy and the environment. At each control step the policy proposes a raw command $a_{\mathrm{raw}}$. The ideal minimum-modification objective is defined as

$$a_{\mathrm{safe}}=arg\underset{a^{\prime }}{min}{∥a^{\prime }-a_{\mathrm{raw}}∥}^2\mathrm{s}.\mathrm{t}.a^{\prime }\in C\left(s\right),$$

(1)

Equation (1) defines the desired minimum-modification objective. The implementation approximates this objective through direct command clipping, line search, and a damped-least-squares tangential correction. Because the admissible set is generally non-convex, exact Euclidean minimality is not claimed. The command $a^{\prime }\in {[-1,1]}^7$ is dimensionless: its first six components are per-joint increments in units of ${\delta }_{\mathrm{arm}}=0.04$ rad, so the commanded joint state is $q^{+}=q+{\delta }_{\mathrm{arm}}{a}_{1:6}^{\prime }$, and the seventh is the gripper command. The admissible set $C\left(s\right)$ is the intersection of eight geometric constraints, four of them coupled through the forward kinematics:

$$\begin{array}{cc}\hfill \left(1\right)\mathrm{joint}\mathrm{limits}\text{:}& q_{\mathrm{lo}}+0.05\le q^{+}\le q_{\mathrm{hi}}-0.05,\hfill \\ \hfill \left(2\right)\mathrm{joint}\mathrm{velocity}\text{:}& |{\dot{q}}_i|\le 2.0\mathrm{rad}/\mathrm{s}\iff |a_i^{\prime }|\le v_{max}\Delta t/{\delta }_{\mathrm{arm}},\hfill \\ \hfill \left(3\right)\mathrm{smoothness}\text{:}& {∥a^{\prime }-a_{\mathrm{prev}}∥}_2\le 1.2(\mathrm{command}\mathrm{units}),\hfill \\ \hfill \left(4\right)\mathrm{workspace}\mathrm{box}\text{:}& p_{\mathrm{lo}}\le p\left(q^{+}\right)\le p_{\mathrm{hi}},\hfill \\ \hfill \left(5\right)\mathrm{reach}\mathrm{sphere}\text{:}& ∥p\left(q^{+}\right)∥\le 0.65,\hfill \\ \hfill \left(6\right)\mathrm{table}\mathrm{Z}-\mathrm{floor}\text{:}& p_z\left(q^{+}\right)\ge -0.15,\hfill \\ \hfill \left(7\right)\mathrm{dynamic}\mathrm{obstacle}\text{:}& ∥p\left(q^{+}\right)-p_{\mathrm{obs}}∥\ge r_{\mathrm{obs}},\hfill \\ \hfill \left(8\right)\mathrm{gripper}\mathrm{sanity}\text{:}& a_7^{\prime }\in [-1,1].\hfill \end{array}$$

If the admissible set is empty, $C\left(s\right)=⌀$, the layer triggers an emergency stop: zero motion, gripper state held.

Constraints (1)–(3) and (8) act directly in command space; (4)–(7) are forward-kinematics-coupled and enforced by a short line search along the command direction to the boundary of the geometric set. Clearance constraint (7) is enforced for the grasp-site, and when a commanded motion would enter the obstacle bubble the layer deflects it tangentially, sliding the tool along the bubble toward the goal rather than merely braking. The deflection maps a desired Cartesian step $v_{\mathrm{def}}$ (inward component removed, with a gentle outward push if already inside) back to joint deltas by a damped-least-squares resolved-rate step [26,27] (chosen over a full inverse-kinematics solve [28] because the layer needs a bounded correction to an existing command rather than a pose)

$$\Delta q=J^{\top }{(J{J}^{\top }+{\lambda }^{2}I)}^{-1}{v}_{\mathrm{def}},$$

(2)

with J the grasp-site Jacobian and $\lambda$ a damping factor; a small buffer above the physical obstacle radius keeps the tool clear of the collision-flag radius even under fast single-step motion. Because (1)–(2) operate purely on realisable commands and geometry, the layer wraps any policy identically at negligible per-step cost. Operating values are listed in Table 2; the whole-arm behaviour behind the tool-point guarantee is audited in Section 5.

3.4. Reactive Shortest-Path Obstacle Detour

On the physical robot the transport phase is executed by a reactive in-plane planner that routes the carried tool around a sensed obstacle along the shortest collision-free path, re-computed at control rate as the obstacle moves. Working in the table plane at the fixed carry height, let $x\in {\mathbb{R}}^2$ be the tool (grasp-site) position, $g\in {\mathbb{R}}^2$ the goal, and let the obstacle be the closed disc $\mathcal{D}=\{y\in {\mathbb{R}}^2:∥y-c∥\le \rho \}$ with sensed centre c and effective radius $\rho$, which throughout this paper abbreviates the enforced radius ${\rho }_{\mathrm{eff}}$ defined in (6) and refined in (14). The straight carry $\overline{xg}$ is admissible when it clears the disc,

$$dist(\overline{xg},c)=\underset{t\in [0,1]}{min}∥x+t(g-x)-c∥\ge \rho ,$$

(3)

in which case the executed path is the segment itself. The minimisation in (3) is over the whole segment, not over its endpoints, and the implementation evaluates it as an exact point-to-segment distance for every consecutive pair of waypoints on the candidate polyline; a path may therefore never pass through the disc between two admissible waypoints.

Tangent–Arc–Tangent Construction:

When (3) is violated, the shortest planar path from x to g avoiding the open disc is the classical taut-string solution: a straight tangent from x to the circle $\partial \mathcal{D}$, a circular arc along $\partial \mathcal{D}$, and a straight tangent from the circle to g. We use it as standard geometry and record only what the safety argument depends on. If the measured tool position lies inside the enforced disc, tangent construction is not used; the filter holds or admits only a recovery command that increases the center distance.

Each end contributes two candidate tangent points, at circle angles $arccos(\rho /d)$ either side of the bearing to that endpoint, and the arc may be swept either way, so the construction yields eight candidates whose total length $L=∥x-T_x∥+\rho \Delta \varphi +∥T_g-g∥$ is available in closed form. The executed detour is the length-minimising admissible candidate,

$${\pi }^{★}=\underset{s_1,s_2,\sigma }{arg\; min}\left[L(s_1,s_2,\sigma )+\eta \mathbb{1}[\sigma \ne {\sigma }_{\mathrm{prev}}]\right]\mathrm{s}.\mathrm{t}.dist(\pi ,c)\ge \rho -ϵ,$$

(4)

where $s_1,s_2\in \{+1,-1\}$ select the tangent side at each end, $\sigma \in \{+1,-1\}$ the arc orientation, the sticky-side penalty $\eta$ discourages flipping that orientation between consecutive control steps (which would otherwise cause limit-cycle chattering), and $ϵ$ is a small numerical tolerance. Every waypoint of ${\pi }^{★}$ is finally pushed out to $\rho +\mathrm{margin}$, so that the discretised segments, not merely the vertices, keep clearance, a distinction that cost us real clearance on hardware (Section 5.2.1).

Live Re-Planning:

For a moving obstacle the plan is recomputed only when the sensed centre has shifted beyond a threshold,

$$∥c_t-c_{\mathrm{plan}}∥>{\delta }_{\mathrm{re}},$$

(5)

and the arc orientation $\sigma$ is carried across replans; together these track a live obstacle while suppressing per-frame re-planning noise.

Latency-Scaled Effective Radius:

The effective radius is not the bare obstacle radius: it absorbs the reaction the layer will need. For a hazard with sensed centre velocity $\dot{c}$, closing speed $v_{\mathrm{close}}=max\left(0,{\dot{c}}^{\top }(x-c)/∥x-c∥\right)$, and worst-case reaction time $T_{\mathrm{react}}$,

$${\rho }_{\mathrm{eff}}=r_{\mathrm{obs}}+d_{\mathrm{mech}}+\underset{\mathrm{reaction}\mathrm{bubble}}{\underbrace{v_{\mathrm{close}}{T}_{\mathrm{react}}}}+\underset{\mathrm{semantic}\mathrm{margin}}{\underbrace{m_{\mathrm{sem}}}},$$

(6)

where $d_{\mathrm{mech}}$ is a fixed mechanical clearance and $m_{\mathrm{sem}}\ge 0$ is the semantic margin supplied by the monitor of Section 3.5 (largest for a person, zero for a confirmed inert object). A stationary, inert obstacle reduces (6) to $r_{\mathrm{obs}}+d_{\mathrm{mech}}$, the small mechanical clearance that makes an in-plane detour feasible in the narrow workspace; a fast or human hazard inflates $\rho$ until no admissible detour exists in (4) and the layer must instead stop.

Routability Budget and Per-Step Guard:

Detouring is only possible while the hazard is slow enough that the required clearance can be recovered within the reaction time. With per-step tool travel $d_{\mathrm{step}}$, required clearance increment $\Delta r$, and the same worst-case reaction time $T_{\mathrm{react}}$, geometric routing can intercept a hazard only when

$$v_{\mathrm{close}}\le {\displaystyle \frac{\Delta r-d_{\mathrm{step}}}{T_{\mathrm{react}}}},$$

(7)

and faster hazards can only be stopped (the quantitative routable/stop split analysed in Section 5.3). Independently of the plan, a hard per-step guard forbids any commanded step $x^{+}$ that would push into the hold radius of the sensing-latency-predicted obstacle $\widehat{c}=c+\dot{c}{\tau }_s$:

$$\mathrm{if}∥x^{+}-\widehat{c}∥<{\rho }_{\mathrm{eff}}+r_{\mathrm{hold}}\mathrm{and}\mathrm{it}\mathrm{decreases}\mathrm{clearance},\mathrm{replace}{x}^{+}\mathrm{by}\mathrm{a}\mathrm{hold},$$

(8)

The constant $r_{\mathrm{hold}}$ is a fixed latency allowance the implementation adds on top of the belief-dependent radius ($0.07$ m for a hazard classified as moving or unknown and $0.035$ m for one classified as stationary) so the guard actually enforced is strictly more conservative than the hold radius alone. Equation (8) is therefore the enforced test, not a relaxation of it, and every clearance figure reported below is measured against the sensed footprint rather than against this inflated test radius. The test rejects inward commanded endpoints under the stated sensing and prediction assumptions up to the sensing horizon ${\tau }_s$, while still allowing the tool to orbit outward when it starts inside the bubble. Equations (3)–(8) are exactly the geometry executed in the hardware detour and hand-freeze trials of Section 5.

3.5. Semantic VLM Monitor

The geometric layer evaluates distance and command constraints, whereas the semantic monitor identifies whether a visible human hand is present in the workspace image. The implemented on-device prompt asks for the number of visible human hands and maps a positive count to human and zero to object; here, object means only that no hand was detected in the queried image. A human verdict triggers a protective hold. The monitor runs asynchronously on a background thread, so the geometric layer continues at control rate while a query is pending. A missing, invalid, or timed-out verdict invokes the configured conservative fallback.

We instantiate this monitor with a genuine backend and measure it on the physical platform, rather than relying on a rule-based stand-in. The backend used in every closed-loop result reported here is on-device: FastVLM-0.5B [29] under the MLX runtime on Apple Silicon, with no network round-trip. The choice is forced by the budget of Section 5.3 rather than by preference (a verdict that arrives seconds after the frame cannot gate a carry that is already in motion) and Section 5.1 shows it returns a verdict in $0.24$ s with $10/10$ hazard frames detected in the pilot image set. Because a $0.5$-billion-parameter model will not reliably answer an abstract safety question, the verdict is posed as a concrete perceptual count (“how many human hands are present?”) and thresholded, rather than as a forced safe/unsafe label.

The monitor is nevertheless backend-agnostic by construction: the same interface accepts hosted vision APIs and local GPU-served models without changing the rest of the pipeline, and we exercise two hosted alternatives (an open-weight Gemma-3-12B-IT through a commercial inference gateway, and Claude Haiku 4.5 through its API) on the identical protocol, sending a downscaled JPEG of the workspace (256 px, quality 55, $\approx 7$ kB) with a fixed safety prompt. All three are evaluated on real RGB frames from the Intel RealSense D435i mounted on the physical NED3 Pro (frames with and without a human hand reaching into the workspace) and the measured reliability and latency of each are reported in Section 5.1. A rule-based fallback, which reasons from the simulator’s safety state rather than pixels, is retained only so that the pipeline and its self-tests remain runnable offline (for example in continuous integration) when no backend is available; it is not used for any reported detection result. The combined layer therefore provides the implemented grasp-site command constraints together with a measured semantic veto, which is the configuration used in the human-hand-avoidance experiments of Section 5.

3.6. Camera-Based Intruder Detection and RGB-D Depth Fusion

The detour of Section 3.4 needs the metric position and size of any intruder, which the fixed Intel RealSense D435i [30] supplies through two complementary channels: a colour image, mapped to world coordinates by a plane homography, and a metric depth map from its active stereo pair. The colour geometry is deliberately minimal. Because the camera is rigidly mounted and every quantity the layer needs lies on the table plane, the image-to-base map is a single plane-to-plane homography [31] fitted from commanded robot poses, an eight-parameter plane-to-plane fit that directly maps image coordinates to the robot-base table plane without requiring a separate hand–eye solve for this planar task [32,33], and whose calibration residual is measurable in the units the layer consumes: millimetres in the base frame rather than reprojected pixels. The mapping is local to the calibrated plane; lens distortion must be corrected separately or treated as part of the measured fit error over the limited workspace. It is valid only on that plane, which is why depth supplies the out-of-plane channel. On the colour channel an intruder is localised by background difference against a reference captured at the start of each carry, then gated by three rules that make it robust to the moving arm: the green task object is removed by colour; any component touching the top of the frame is discarded as the robot body, which is always attached there in this fixed view whereas a table intruder is not; and only components whose mapped centre lies inside the calibrated workspace survive, rejecting off-table clutter and cabling. The largest survivor gives the centre c and a footprint radius, inflated by 8 mm so the estimate is conservative.

When depth is available, the two modalities are fused for confirmation rather than for coordinates: a colour detection is accepted when it matches a depth foreground region, an unmatched depth-only foreground is confirmed over three consecutive frames and is treated as stop-only (never routed around on depth alone), and the colour detector remains as a fall-back if the depth node drops out. World $x,y$ always come from the colour homography; depth adds complementary range-based confirmation. Releasing the object additionally requires the drop column to read clear in both colour and depth whenever depth was available at background capture. This is the standard sensor-fusion redundancy argument (the two sensing channels must agree before the tool is routed close to an obstacle) and in practice it is what removes the occasional sub-centimetre grazes that the colour-only detector allowed at the footprint edge (Section 5.2.2). Because depth-to-colour registration on the low-bandwidth on-robot link is only an approximate factory-field-of-view map, depth is used for confirmation and range gating, not to override the calibrated colour geometry.

3.7. Task Policies

Two task policies are driven through the identical safety layer. On the physical robot the transport phase is executed by the reactive in-plane planner of Section 3.4. In simulation the layer is additionally exercised behind a scripted RRT planner and behind a learned Action-Chunking Transformer (ACT) [18] trained by imitation on planner demonstrations, which is what establishes that the layer is indifferent to the policy above it. The learned policy, the sudden dynamic-obstacle stress test, the training procedure, the simulation evaluation metrics and every simulation result are reported in full in the Supplementary Material; the learned policy is not transferred to hardware, so no claim in this paper rests on it.

3.8. Implementation Parameters

For reproducibility, Table 2 lists the operating parameters of the safety layer, the reactive planner and AEGIS exactly as used in every reported experiment. The learned policy’s architecture, training schedule and dataset are given in the Supplementary Material. The production ACT policy is the first self-imitation round: it is trained on 989 successful demonstrations (840,650 steps) composed of 495 self-imitation rollouts, 273 planner sudden-obstacle demonstrations, and 221 static-obstacle demonstrations, with the demonstration regimes mixed 60/25/15 (sudden/static/clear) and the grasped-object shape randomised per episode. A second self-imitation round and a higher-capacity network were both evaluated and rejected (Section 5).

4. AEGIS: Age-Aware, Evidence-Guarded Intervention for Safety

The semantic margin $m_{\mathrm{sem}}$ in the effective radius (6) is where the two layers meet and where the age-dependent rule is introduced. A semantic verdict is produced only every $\Delta$ seconds ($\approx 0.24$ s on-device); between verdicts it is stale, and the scene may change inside that blind window. The two obvious ways to handle this are both inadequate, and in opposite directions: always assuming a person, so the margin is permanently the full human bubble and the layer over-stops on every benign object; or trusting the latest verdict at face value, so the layer routes at object clearance even in the window after a hand has entered but before the next verdict corrects it. Both are recovered below as special cases of one family, and the reason each is a natural thing to build (and therefore worth comparing against) is set out with the family in Table 3. We resolve the dilemma with a margin that is a function not only of a semantic belief but of its age: an age-aware, evidence-guarded intervention for safety (AEGIS).

Belief Dynamics:

Per tracked intruder i we maintain a belief $p_i\in [0,1]$ that it is a person. Let $t_k^{\mathrm{img}}$ be the capture time of the source image, $t_k^{\mathrm{arr}}$ the time at which its verdict becomes available to the controller, and $v_k\in \{human,object\}$ the corresponding verdict. The age of information used by AEGIS is $\tau =t-t_k^{\mathrm{img}}$ for $t\ge t_k^{\mathrm{arr}}$. When the verdict first becomes available, its age already includes image acquisition, queuing, preprocessing, and inference delay.

The implementation does not yet realise this definition: the monitor stamps a verdict when inference returns and the controller adopts it on the next tick, so the enforced age runs from $t_k^{\mathrm{arr}}$, short of the true age by $0.24$–$0.37$ s here. As ${\rho }_{\mathrm{eff}}$ increases in $\tau$, the error is unsafe-side. After a fresh object verdict ($\widehat{p}=0.033$), ageing by the missing interval raises p by $0.279$–$0.370$, under-sizing the radius by $22.3$–$29.8$ mm at the simulation constants ($R_{\mathrm{human}}-d_{\mathrm{mech}}=80$ mm) and by $13.4$–$17.9$ mm at the platform-scaled hardware constants (48 mm). Neither buffer covers its own worst case: ${\beta }_{\mathrm{epi}}=25$ mm leaves $4.8$ mm uncovered in simulation, and 12 mm leaves $5.9$ mm uncovered on the arm, where even the best case at $0.24$ s exceeds the buffer by $1.4$ mm. Timestamping at capture is a one-line change that removes the term, and the reported runs predate it. Between verdicts no new evidence arrives, so the belief must not stay where the last verdict left it: it relaxes toward a cautious prior $P_0$ at a rate set by $T_{\mathrm{dec}}$,

$${\displaystyle \frac{\mathrm{d}{p}_i}{\mathrm{d}\tau }}={\displaystyle \frac{P_0-p_i}{T_{\mathrm{dec}}}},p_i\left(0\right)={\widehat{p}}_i,$$

(9)

whose solution is the exponential relaxation used at run time,

$$p_i\left(\tau \right)={\widehat{p}}_i+(P_0-{\widehat{p}}_i)\left(1-e^{-\tau /T_{\mathrm{dec}}}\right)=P_0+({\widehat{p}}_i-P_0)e^{-\tau /T_{\mathrm{dec}}}.$$

(10)

The second form makes the two limits explicit: $p_i\left(0\right)={\widehat{p}}_i$, the verdict just received, and $p_i\left(\tau \right)\to P_0$ as $\tau \to \infty$, so an indefinitely stale verdict carries no authority and the layer falls back on the prior rather than on the last thing it happened to see. The post-verdict value ${\widehat{p}}_i$ is a reliability-weighted update rather than a hard assignment, so that a single misclassification cannot drive the belief to a certainty the monitor does not have,

$${\widehat{p}}_i\leftarrow (1-\eta )p_i\left(t_k^{\mathrm{arr}-}\right)+\eta \mathbb{1}[v_k=human],\eta \in (0,1],$$

(11)

with $\eta$ the trust placed in one verdict ($\eta =0.95$ here). Finally, association is not assumed: if the detector reports a blob whose centre or radius jumps by more than ${\delta }_{\mathrm{blob}}$ between frames, it may be a different physical object wearing the old track’s identity, so the belief is reset,

$$\parallel c_i\left(t\right)-c_i\left(t^{-}\right)\parallel >{\delta }_{\mathrm{blob}}\mathrm{or}|r_i\left(t\right)-r_i\left(t^{-}\right)|>{\delta }_{\mathrm{blob}}⟹p_i\leftarrow P_0,\mathrm{cached}\mathrm{verdict}\mathrm{invalidated}.$$

(12)

A new or reassociated blob therefore cannot inherit the previous object’s object verdict.

From Belief to Distance:

The belief scales the semantic margin between the small mechanical clearance owed to a confident inert object and the full protective distance owed to a person, plus a fixed epistemic buffer ${\beta }_{\mathrm{epi}}$ retained from any routed object because one classifier is never fully trusted:

$$m_{\mathrm{sem}}\left(\tau \right)=p_i\left(\tau \right)\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right)+{\beta }_{\mathrm{epi}}.$$

(13)

Substituting (13) into (6) gives the radius the layer actually enforces, with every term carrying a distinct physical meaning:

$${\rho }_{\mathrm{eff},i}\left(\tau \right)=\underset{\mathrm{footprint}}{\underbrace{r_i}}+\underset{\mathrm{mechanical}}{\underbrace{d_{\mathrm{mech}}}}+\underset{\mathrm{kinematic}}{\underbrace{v_{\mathrm{close}}{T}_{\mathrm{react}}}}+\underset{\mathrm{semantic}}{\underbrace{p_i\left(\tau \right)\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right)}}+\underset{\mathrm{epistemic}}{\underbrace{{\beta }_{\mathrm{epi}}}}.$$

(14)

Relation to Standard Separation-Distance Concepts:

Equation (14) combines five design terms: the estimated footprint radius, a mechanical clearance, a reaction allowance, an age-dependent semantic term, and an epistemic buffer. This decomposition is informed by speed-and-separation concepts used in collaborative-robot safety [2,12]; however, the present prototype is neither an implementation nor a certification of ISO/TS 15066. In particular, its perception chain, timing measurements, and semantic classification are not safety-rated. Within the proposed model, setting $p_i\equiv 1$ gives the class-agnostic human-margin endpoint, whereas the age-dependent score allows the margin to vary between the algebraic endpoints defined below.

Only the semantic term depends on time. Between two consecutive verdicts, substituting (10) gives

$${\displaystyle \frac{\mathrm{d}{\rho }_{\mathrm{eff},i}}{\mathrm{d}\tau }}={\displaystyle \frac{(P_0-{\widehat{p}}_i)\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right)}{T_{\mathrm{dec}}}}{e}^{-\tau /T_{\mathrm{dec}}},$$

(15)

so the radius is monotone between verdict updates, but its direction depends on the latest update. After an object verdict with ${\widehat{p}}_i<P_0$, the radius increases toward the cautious prior as the evidence ages. After a human verdict with ${\widehat{p}}_i>P_0$, it decreases toward the same prior without falling below it. Since $p_i\left(\tau \right)\in [0,1]$, the implemented radius satisfies

$$r_i+d_{\mathrm{mech}}+v_{\mathrm{close}}{T}_{\mathrm{react}}+{\beta }_{\mathrm{epi}}\le {\rho }_{\mathrm{eff},i}\left(\tau \right)\le r_i+R_{\mathrm{human}}+v_{\mathrm{close}}{T}_{\mathrm{react}}+{\beta }_{\mathrm{epi}}.$$

(16)

These are algebraic endpoints of the implemented radius and should not, by themselves, be interpreted as certified safety or closed-loop liveness limits.

The Two Degenerate Policies, and Where Each Comes from:

The reference points we compare against in Section 5.2.3 are exactly the two endpoints of (16) made permanent (Table 3). They are not strawmen invented for the comparison; each is the policy one arrives at from an established engineering position, and between them they cover the two ways a practitioner would actually wire a semantic monitor to a safety layer.

Worst-case fixes $p_i\equiv 1$: the intruder is treated as a person at all times. Its lineage is classical machine safety. A light curtain, a safety-rated laser scanner, or speed-and-separation monitoring under ISO/TS 15066 [2,12] is deliberately class-agnostic (it registers that something has entered the cell but not what that something is) so the only defensible margin is the one that would protect against the worst hazard the cell admits. Worst-case is therefore what current certified practice prescribes in the absence of a semantic channel, and by (14) it is the $p\equiv 1$ member of our own family, not a rival to it. Its defining property is that it does not consult the monitor at all: remove the vision–language model from the system and worst-case behaves identically.

Trust-latest fixes $p_i\equiv \mathbb{1}[v_k=human]$, with no dependence on $\tau$, while retaining the same epistemic buffer as AEGIS in the primary comparison. Its lineage is the standard perception–control composition: a perception module emits an estimate and the controller consumes it as the current state of the world, exactly as it would a pose or a depth reading. Applied to a classifier this is the natural way to bolt a modern semantic monitor onto an existing safety filter, and it is what one obtains by treating a discrete, intermittently refreshed label as though it were a continuously available signal. The step is so natural that the assumption it smuggles in (that the label is still true when it is used) is rarely made explicit.

What the two share is the defect that matters here: both are constant in $\tau$. Worst-case is pinned at the upper bound of (16) and never leaves it; trust-latest jumps between the two bounds when a verdict arrives but is flat in between, so throughout any blind window it is exactly as safe, or as unsafe, as the verdict that opened that window. Neither can move within the bounds as evidence ages (Figure 3), which is precisely why neither can be safe and live at once, and why their failures are complementary rather than merely different: worst-case pays for safety with liveness on every benign object, trust-latest pays for liveness with safety in exactly the window the construction is designed to cover. Section 5.2.3 measures both prices. We now state what the construction buys, and under which assumptions.

Assumption A1. 

Whenever a true human intrusion is present and robot motion is permitted, the implemented score satisfies $p_i\left(\tau \right)\ge \underline{p}$ for a specified $\underline{p}\in [0,1]$. A track change invokes a fallback that resets the score to the cautious prior $P_0$, so the bound is preserved whenever $\underline{p}\le P_0$.

Assumption A2. 

The estimated footprint contains the relevant planar intrusion, the inward closing speed is bounded by $v_{\mathrm{close}}$, the acquisition-to-completed-command interval is bounded by $T_{\mathrm{react}}$, and the executed grasp-site command satisfies Equation (8).

Proposition 1. 

Under Assumptions A1 and A2, if

$${\beta }_{\mathrm{epi}}\ge (1-\underline{p})\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right),$$

(17)

then, at each guarded decision and within the stated planar and reaction bounds, the post-reaction distance from the grasp site to the estimated footprint boundary is at least $R_{\mathrm{human}}$.

Proof. 

At command acceptance, Equation (14) and the guard require a center distance of at least

$$r_i+d_{\mathrm{mech}}+v_{\mathrm{close}}{T}_{\mathrm{react}}+p_i\left(\tau \right)\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right)+{\beta }_{\mathrm{epi}}.$$

Subtracting the footprint radius and the bounded inward motion leaves at least

$$d_{\mathrm{mech}}+p_i\left(\tau \right)\left(R_{\mathrm{human}}-d_{\mathrm{mech}}\right)+{\beta }_{\mathrm{epi}}.$$

Using $p_i\left(\tau \right)\ge \underline{p}$ and Equation (17) makes this quantity no smaller than $R_{\mathrm{human}}$.    □

Proposition 2. 

If verdicts are fresh and confidentlyobject($\widehat{p}=0$, $\tau \to 0$), then $m_{\mathrm{sem}}\to {\beta }_{\mathrm{epi}}$ and ${\rho }_{\mathrm{eff}}$ converges to the radius of a semantics-free geometric layer inflated by ${\beta }_{\mathrm{epi}}$; consequently every task the semantics-free layer completes is also completed under AEGIS whenever a clearance margin of ${\beta }_{\mathrm{epi}}$ is available. This is a path-feasibility statement, not a closed-loop liveness guarantee.

Proof. 

With $\widehat{p}=0$, (10) gives $p\left(\tau \right)=P_0\left(1-e^{-\tau /T_{\mathrm{dec}}}\right)\to 0$ as $\tau \to 0$; hence $m_{\mathrm{sem}}\to {\beta }_{\mathrm{epi}}$ by (13) and ${\rho }_{\mathrm{eff}}\to r+d_{\mathrm{mech}}+v_{\mathrm{close}}{T}_{\mathrm{react}}+{\beta }_{\mathrm{epi}}$, which is exactly the semantics-free radius plus ${\beta }_{\mathrm{epi}}$. Any path admissible for the semantics-free layer with ${\beta }_{\mathrm{epi}}$ of spare clearance is therefore admissible under AEGIS.    □

Condition (17) is a design requirement. The pilot semantic-monitor data do not establish a valid lower bound $\underline{p}$ during every true human intrusion, particularly before the first updated human verdict. Assumption A1 states only the fallback the implementation provides: a track change resets the score to $P_0$ as assumed, while a missing or stale verdict is aged by (10) rather than held, since the monitor is initialised to object and the staleness halt is guarded on a verdict being absent, which it never is. Both are one-line fixes, neither was in place for the reported runs. The proposition therefore neither certifies the monitor nor shows that the deployed constants satisfy the condition, and the gap is quantifiable. Rearranged, (17) requires $\underline{p}\ge 1-{\beta }_{\mathrm{epi}}/(R_{\mathrm{human}}-d_{\mathrm{mech}})$: $0.69$ in simulation (25 mm, 80 mm) and $0.75$ on the arm (12 mm, 48 mm). Neither is reached, since $P_0=0.65$ and a confident object verdict drops the belief to $\widehat{p}=0.033$; the true infimum would demand 77 and 46 mm of buffer, three to four times what was deployed. The physical trials therefore ran below the sufficient condition, and their outcome is empirical rather than an instance of the guarantee.

Remark 1. 

Assumptions A1 and A2 are deployment obligations rather than properties established by the present experiments. The pilot semantic-monitor data do not validate the lower bound $\underline{p}$, and the geometric result is limited to the estimated planar footprint and constrained grasp site. Proposition 1 therefore does not certify the semantic monitor or the complete robot.

AEGIS applies age-dependent semantic evidence to the hold radius of the present safe-action architecture. Related work has considered age of information in delayed safety control and staleness-aware asynchronous VLM regulation [10,11]. Section 5.2.3 evaluates the specific AEGIS construction against the two endpoint policies used in this study.

5. Test and Results

The results are ordered by evidence tier rather than by chronology. The physical robot comes first (the semantic monitor measured on real frames, online re-routing around a human hand, the closed-loop route-versus-freeze contrast, the AEGIS policy comparison under the live verdict stream, and the measured latency budget) because these are the claims the architecture stands on. The simulation campaign is summarised in Section 5.4 and reported in full in the Supplementary Material, where it supplies the large-sample statistics and the policy comparison that would each require hundreds of physical episodes. Every result names the tier of Table 1 that produced it.

5.1. Real-Robot Semantic Monitor

To move the semantic monitor from an architectural placeholder to a measured component, we ran the hosted backend of Section 3.5 (Gemma-3-12B-IT) on real RGB frames from the D435i mounted on the physical NED3 Pro, each sent as a small JPEG with a fixed safety prompt. The set contains 10 frames of the normal workspace (correct verdict safe) and 10 in which a hand or forearm reaches in next to the gripper (correct verdict stop). The monitor is correct on all 20: it flags every hazard frame ($10/10$) (95% Wilson interval: $72.2$–$100\%$) and never freezes on a safe one ($0/10$) (95% Wilson interval: 0–$27.8\%$), at a mean query latency of $1.6$ s (95th percentile $4.1$ s), consistent with an asynchronous keyframe check rather than a hot-loop call. The verdicts name the hazard rather than reporting a distance (“a human hand is present in the workspace near the robot arm”), which is the property the geometric layer cannot supply. This is a small single-scene set, sufficient to establish that a real backend detects the target hazard at a practical latency; a multi-subject, multi-viewpoint campaign is left as future work.

The monitor is backend-agnostic: swapping the hosted open-weight model for Claude Haiku 4.5 through the Anthropic API is a configuration change only, with no pipeline modification. Table 4 repeats the identical 20-frame protocol on both backends. Both preserve the safety-critical property (every hand-in-workspace frame is flagged ($10/10$ recall)) while the Claude backend stops on $2/10$ safe frames where the open-weight model stops on none; those errors lie in the conservative direction in this test, but the small image set does not establish backend-level safety. This supports treating the semantic monitor as a replaceable component of the architecture, with the latency-budget analysis of Section 5.3 applying to whichever backend is mounted.

5.1.0.9. Monitor Specification:

The deployed on-device monitor is mlx-community/FastVLM-0.5B-bf16 [29] run through MLX $0.32.0$ and mlx-vlm$0.6.5$ on an Apple M4 Pro (48 GB, Metal, no CUDA). Frames are down-sampled to a 256 px longest side and decoding is capped at six tokens at temperature 0; the same verdicts are obtained at 512 px in the same $0.24$ s, so the latency is vision-encode bound rather than resolution bound. The prompt is a count rather than a label (“How many human hands do you see in this image? Answer with a number.”) because a forced binary label collapses a model of this size onto a single class, whereas a concrete count does not; the reply is parsed by taking the first integer in the string, falling back to a small number-word table. A count of zero yields safe and any count of one or more yields stop, which is also what supplies the human-versus-object split the geometric layer consumes. There is no per-query timeout: the monitor runs in its own thread on a $0.6$ s period and the controller always reads the most recent completed verdict, so a slow query ages the evidence rather than blocking the loop.

The latency figures come from 20 timed queries, one per frame of the $10+10$ pilot set, warm-up excluded: mean $0.236$ s, median $0.237$ s, 95th percentile $0.238$ s, minimum $0.233$ s, a spread of 5 ms, which is why a single figure of $0.24$ s is quoted throughout. That set is a development set, not an independent test set: the same twenty frames were used to choose the counting prompt over a forced binary label and to select FastVLM over a smaller candidate, so the reported $20/20$ accuracy is an upper bound on what an unseen set would give. The latency, which does not depend on the labels, is unaffected by this; the accuracy should be read as a feasibility check rather than as a measured error rate.

Two properties of this parser are not conservative. An unparseable reply (no digit, no recognised number word) scores zero hands and therefore safe, as does an exception in the monitor thread, after which the controller keeps reading the previous verdict. The semantic channel thus fails permissive, and the geometric layer of Section 3.3 is what keeps the behaviour safe: the hazard is still detected as a blob, and an unconfirmed depth-only foreground remains stop-only whatever the verdict. Second, counting hands makes an arm with no hand in frame semantically invisible. The pilot set contains hand-and-forearm reach-ins (recall $10/10$) but does not isolate that case, so no claim is made for it; geometrically such an intruder is still detected and routed around or held.

The third backend, FastVLM-0.5B, runs entirely on-device (Apple Silicon, MLX runtime) with no network round-trip, and returns the verdict in $0.24$ s on average (95th percentile $0.25$ s) (roughly $7\times$ faster than the hosted open-weight model and $11\times$ faster than the hosted API) while flagging all 10 hazard frames in this pilot set and zero false stops on this set. The latency is dominated by the vision encoder rather than by decoding: a smaller SmolVLM-256M was both slower ($0.66$ s) and non-discriminating, so the efficient encoder, not raw parameter count, is what breaks the half-second barrier; posing the verdict as a concrete perceptual count (how many human hands are present?) rather than a forced binary label was necessary for the small model to answer reliably. This directly affects the latency budget of Section 5.3: a semantic verdict at $0.24$ s rather than seconds shrinks the required safety bubble for a moving hazard by an order of magnitude, moving the semantic layer from strictly quasi-static hazards toward slowly-moving ones, while the hard real-time reaction to fast hazards still belongs to the geometric layer.

5.2. Closed-Loop Validation on the Physical Robot

All hardware results in this paper come from the same overall pipeline, with session-specific calibration and sensing configurations stated for each trial: the fixed RGB-D camera of Figure 2, the calibrated image-to-table homography, the workspace-gated intruder detector, the on-device semantic monitor, and the geometric projection layer, running closed-loop at control rate. What varies between the experiments below is only the hazard presented to that pipeline (a human hand, an inert obstacle, or a hazard whose semantic class the monitor must resolve) and, in the last experiment, how the hold radius is derived from the verdict. They are three views of one system, not three methods.

5.2.1. Re-Routing Around a Human Hand

Before adding semantics we exercise the geometric layer alone, on the claim that it (not the semantic monitor) owns moving hazards. The object pose comes from the calibrated homography of Figure 4. Two calibration sessions are used and they are not interchangeable: the 22 July session behind Figure 5 and Figure 6 fits nine points to a mean of $0.92$ mm (worst $1.45$ mm), while the 30 July session behind Figure 7 and Figure 8 was re-fitted as the camera was re-aimed and loads an eight-point free-standing-slab homography at mean $2.36$ mm (worst $3.82$ mm, one $11.9$ mm outlier rejected). Both are fitting residuals on the correspondences used to solve the homography; no held-out set was collected. The hazard pose comes from a cheap skin-colour detector at 30 Hz supplying the operator’s hand as a live obstacle with a 7 cm bubble; no vision–language model is involved. Running the identical corridor twice, the clear workspace gives a straight $18.4$ cm carry in 9 commands, while with the hand on the carry line (where the straight path would pass $1.5$ cm from the hand centre, deep inside the bubble) the layer deflects around it: $29.2$ cm of path ($+59\%$), one detour, a minimum tool–hand distance of $\mathbf{9}.\mathbf{6}$ cm and zero bubble breaches, still delivering ($1.8$ cm from target, against $0.9$ cm in the clear run). Withdrawing the hand mid-carry re-plans immediately back to the straight line, so the detour cost is paid only while the hazard is present.

Two failure modes appeared here that the simulated study did not surface, and both bear on the safety-filter comparison reported in the Supplementary Material. First, re-planning from scratch at every command (executing only the first waypoint and re-choosing the detour side each time) made the tool oscillate between deflection and retreat for 40 commands without net progress: the reactive rule stalled in a local minimum, precisely the pathology that costs the pointwise CBF filter its task completion, here observed live. Committing to a plan and re-planning only when the hazard moves more than 3 cm, with a hysteretic detour side, removes it. Second, placing the via-point outside the bubble does not keep the path outside it: the densified segments reaching the via cut to $5.8$ cm. Projecting every commanded waypoint radially out to the bubble radius, with a hard pre-command guard, restores the $9.6$ cm. The guarantee must therefore be enforced on the executed command sequence rather than on the plan’s landmarks, exactly the discipline the projection imposes at every step. Two caveats bound this trial: the hand was already on the carry line when transport began, so it demonstrates re-routing around a present hazard rather than reaction to one appearing mid-motion, and a single matched pair establishes the behaviour, not its statistics.

5.2.2. Routing an Object versus Freezing for a Person

The re-routing trial above deflects around a hand that is treated purely geometrically. We now close the full loop on the physical NED3 Pro: the reactive shortest-path detour of Section 3.4 and the semantic monitor of Section 5.1 run together on every carry, and the monitor’s verdict alone selects the response. The fixed Intel RealSense D435i drives a plane homography re-fitted at the start of the session (mean re-projection residual $0.92$ mm, worst $1.45$ mm, over nine points), and the object is carried along the largest in-plane path between two workspace corners ($\approx 24$ cm). The semantic verdict is produced on-device by a $0.24$ s vision–language model (Section 5.1), so it refreshes fast enough to gate a moving carry. Every trial is recorded with a colour decision-narration overlay and logged per control tick; representative frames appear in Figure 5Figure 6 and the trials are summarised in Table 5.

The same code produces opposite behaviours depending only on the semantic verdict (Figure 5). The sharpest evidence is a single uninterrupted carry in which the intruder on the line is alternated between a box and an operator’s hand while the arm is moving, so the two behaviours are separated by seconds rather than by a change of run. Whenever the monitor returns object, the geometric layer executes the tangent–arc–tangent detour of Section 3.4, leaving the straight line by 4–7 cm and holding a positive tool–obstacle clearance throughout, minimum $5.9$ cm over the carry, never negative (Figure 5d, f, g). Whenever it returns human, the layer freezes: over six such windows totalling 131 control ticks ($16.0$ s) with the hand present, the commanded tool motion is $0.0$ mm in every one (the arm never advances toward the person) and each time the hand is withdrawn the carry resumes on its own, unprompted (Figure 5b, h). The same run also exercises the endpoint guard: with the destination itself occupied, release is withheld until eight consecutive frames prove it clear (Figure 5e), after which the object is delivered. Routing around a benign object and holding away from a person are thus one policy under two verdicts, here within a single carry. The isolated single-verdict runs reported in Table 5 give the same picture: $3.6$ cm of detour at $4.0$ cm minimum clearance under object, and $0.0$ mm over 127 consecutive ticks under human, which at the logged $7.9$ Hz control rate is $16.1$ s.

The detour also tracks a live, moving obstacle. When a hand-held object is slid along the path (sensed centre traversing $6.9$ cm), the layer continuously re-plans (Equation (5)): over the carry it spends 69 control ticks routing and 26 holding, keeps a minimum tool–obstacle clearance of $3.3$ cm with zero moving overlap, and delivers. Because the operator’s hand is intermittently in frame while pushing the object, the monitor also correctly triggers brief human freezes during the same run, human safety pre-empting the object detour, exactly as intended.

Adding a metric depth channel from the same D435i tightens the margin further. With RGB-D fusion the depth-confirmed detours held strictly positive moving clearance throughout every carry (minimum $1.9$–$3.05$ cm, with no negative-clearance control tick recorded), including the combined run of Figure 5, in which the layer both routes around an object and freezes for a hand within the same carry. Figure 6 shows the fusion itself: the colour channel proposes the hazard first, and the detour is committed only at the control tick on which a depth detection matches it, the enforced radius then being the larger of the two estimates. Where no route survives at all, the layer holds instead of squeezing past (Figure 6b), and the carry’s minimum clearance is $+2.9$ cm with no negative tick. Depth thus both supplies a second, complementary sensing modality and removes the sub-centimetre grazes that the colour-only detector occasionally allowed at the footprint edge.

A continuous 30-carry autonomous run delivered $30/30$; scenes with up to three simultaneous obstacles were delivered (minimum moving clearance $3.5$–$4.7$ cm) or brought to a protective hold when (4) admitted no detour. Long carries under the raised-camera homography landed within $0.69$–$0.93$ mm over 147–269 mm paths, consistent with the $1.8$ mm grasp repeatability measured at calibration. During development, before the detour margin was enlarged and the stale-track and occlusion-persistence guards were added, a few runs recorded a transient moving overlap or a place-phase contact; these count as failures, an offline replay under the corrected guards holds every previously-executed unsafe command, and all numbers above are from the corrected system.

The rows in Table 5 come from several demonstration protocols and should not be pooled as independent repetitions of one experiment. Clearance is calculated against the system’s estimated footprint rather than independent motion-capture ground truth.

5.2.3. AEGIS Versus the Two Degenerate Policies

We evaluate whether a margin indexed by verdict age improves the observed clearance–availability balance relative to the two endpoint policies in Table 3. We test that claim against the two degenerate alternatives of Table 3: worst-case, which never consults the verdict and always applies the full human distance, and trust-latest, which applies the latest verdict at face value however old it is. Because they are the two endpoints of (16) held fixed, an outcome in which each fails on a different axis is the signature the construction predicts, not a coincidence of tuning. The intended matched comparison holds the geometric layer, detour, guard, reaction term, and epistemic buffer fixed while varying only the rule for $p_i$. Table 6 carries out that comparison at the kinematic tier.

The comparison is run at two evidence tiers (Table 1). The comparison begins kinematic: the tool is a point tracking a planned path and the intruder a disc with a scripted appearance, motion and true class, because the quantity under test is the timing relationship between verdict age and margin, not manipulator dynamics; each scenario is repeated over 250 randomised seeds at this tier. We then corroborate it in full simulation on the 6-DoF arm, and finally on the physical robot; Table 7 collects all three tiers.

The two baselines fail in exactly the complementary ways the construction predicts: worst-case never delivers the benign static obstacle because the full human bubble permanently blocks the corridor, while trust-latest delivers it but routes into $12\%$ of the fast hand reach-ins, because it is still acting on an object verdict emitted before the hand arrived. In the reported kinematic runs, AEGIS delivered $249/250$ static-obstacle episodes and no reach-in overlap was observed across the tested seeds. To separate the two effects that differ between AEGIS and the baselines (the age-indexed belief and the epistemic buffer, which only AEGIS received) we re-ran the kinematic tier with the buffer as an explicit factor, holding the geometric layer, detour, guard and reaction term fixed and varying only the rule for $p_i$. Table 6 reports the semantic-switch scenario, in which a confirmed object’s true class flips to human inside the blind window; it is the case the mechanism is designed for and the only one of the four that discriminates on safety.

Read across both scenarios, the table places AEGIS between the two endpoints rather than above them. Worst-case is the safest rule wherever holding is free (in semantic-switch the intruder eventually leaves, so waiting costs nothing and it scores $100\%$) but in sudden-static the obstacle persists and the same rule delivers $0\%$, which is the price the construction predicts. AEGIS pays neither: it delivers every episode of the persistent case while remaining collision-free there, and on the switching case it sits near trust-latest rather than near worst-case. The mechanism claim is therefore about the trade-off, not about dominating either endpoint.

Within that trade-off, the buffer-matched comparison is what isolates the rule. With ${\beta }_{\mathrm{epi}}=0$ for every policy the age-indexed rule is significantly safer than trusting the latest verdict, $41\%$ against $27\%$ ($p=2.7\times {10}^{-4}$); this is the unconfounded evidence for the mechanism. Granting the same buffer to both closes the gap ($19\%$ against $20\%$, $p=0.78$): a wider radius keeps the tool longer in the slow-approach band beside the hazard, so it is exposed across more ticks when the class flips, and the margin it buys is paid for in dwell time. The configuration reported elsewhere in this paper (AEGIS with the buffer, baselines without) is therefore the one setting that mixes the two effects, and the comparison it supports is between AEGIS as deployed and the baselines as deployed, not between the belief rules alone. Finally, no constant belief is competitive on both axes at once: values that still route reach at most $1\%$ human-safe, and raising the constant to buy margin costs delivery instead, down to $25\%$ at $p=0.49$. With a hold threshold in the family a constant belief can only stop always or never; the age-indexed rule is the only one that crosses the threshold within a carry, which is what indexing by age buys.

We then re-run the three policies on the full 6-DoF arm in MuJoCo, with contacts and the object rigidly carried, using the identical AEGIS belief and constants but now setting the geometric layer’s hold radius from the belief every control step of a real pick-and-place carry past a central intruder (30 seeds). The intruder is reported as an object, and we measure the counterfactual a single mislabel poses: were that object in fact a person, would the moving tool have stayed outside $R_{\mathrm{human}}$? The ordering reproduces on the physical-scale arm, but the separation is tighter than in the kinematic idealisation, because the nominal routing clearance already sits near $R_{\mathrm{human}}$: AEGIS raises the human-safe fraction from $47\%$ to $73\%$ at identical delivery rather than to $100\%$, and its $10\%$ delivery residual is the planner’s placement descent stalling on a few seeds, common to both live policies, not a routing failure. This shows a partial improvement on the 6-DoF model: AEGIS keeps a human-respecting margin from an object a naive policy would skim, without giving up throughput.

On the Physical Robot:

Finally, the same three policies were run on the physical NED3 Pro, which raises AEGIS to the hardware evidence tier and makes it the only component of the architecture validated at all three (Table 1). The belief is driven by the live on-device FastVLM verdict stream described in Section 5.1 ($\approx 0.39$ s per verdict in this session), and the resulting hold radius is written into the geometric layer at every control tick of a real $22.4$ cm carry, with a box placed on the carry line as the inert intruder. Only the derivation of the radius differs between policies; the arm, scene, detector and carry are identical.

The protective distances are scaled to the platform ($0.06/0.012/0.012$ m against $0.10/0.020/0.025$ in simulation) for a geometric rather than a presentational reason: a $22.4$ cm carry with a $\approx 3$ cm intruder cannot admit a 10 cm human bubble, under which every policy (including AEGIS) simply holds and the comparison becomes vacuous. The scaling preserves the ordering; it does not create it.

The small physical sample shows the same qualitative ordering, with wide uncertainty due to the limited and unequal run counts. Worst-case never moved while the intruder was present (zero moving ticks across 1563 hazard ticks) and when the box was genuinely left in place for a full run it never delivered at all, exhausting the tick limit; this is the $0\%$ delivery of Table 7 realised physically. Trust-latest delivered in all six of its runs, but in one of them drove the tool inside the detected obstacle footprint (a minimum edge clearance of $-5.6$ mm): its radius collapses to $3.0$ cm on a fresh object verdict and rises to $14.8$ cm only once the monitor sees a hand, by which time the tool has already passed. AEGIS kept positive internally measured edge clearance in the three reported runs (worst $+19.7$ mm) while still delivering, and in the matched first round it was also the fastest of the three ($25.9$ s against $26.1$ s and $65.9$ s).

The mechanism itself is directly observable in the logged radius rather than inferred from outcomes: during the AEGIS runs the effective radius breathes with the belief ($5.2\to 9.9$ cm in one run and $5.4\to 16.0$ cm in another) expanding as a verdict ages or turns human and collapsing again on a fresh, confident object. In the cleanest trust-latest run it instead sat pinned at $4.0$–$4.2$ cm throughout.

Figure 7Figure 8 show the two policies at the instants their logs identify as decisions, and make the difference visible rather than statistical. In the AEGIS run the same scene produces two different geometries within nine seconds: with a human verdict the hold radius stands at 158 mm and the arm is stationary; once a fresh object verdict arrives it collapses to 56 mm and the planner commits to a detour arc around the box. In the trust-latest run the radius is pinned at 30 mm by the same object verdict and the tool ends up inside the detected footprint, the negative clearance of Table 7. The per-tick log attributes that event more precisely than the frames. The intruder track was lost for four consecutive ticks ($0.5$ s); with no hazard present the layer correctly saw a clear line and resumed the carry; and when the blob was re-acquired the tool was already $6.7$ mm inside it. On all three negative ticks the clearance was increasing (the tool was withdrawing, never advancing) so the per-step guard of (8) never admitted an inward command. The magnitude also has to be read against the calibration it is measured in: the $-5.6$ mm figure comes from a session whose worst fitting residual is $3.82$ mm, so the defensible result is the sign (trust-latest crosses the detected footprint and AEGIS does not), not the exact depth of the crossing. The event is therefore a detection dropout that the small radius left no room to absorb, not a commanded incursion, and it is evidence about the coupling between margin and perception gaps rather than about verdict trust alone. Dropouts of this kind occur in every run of the session, including the AEGIS runs, at 6–$12\%$ of carrying ticks; with three and six runs per policy we cannot say whether the wider radius absorbed them or whether none happened to coincide with a close approach.

Two qualifications attach to that reading. Clearance is measured over the ticks on which a policy commands motion; counting protective holds turns four further runs momentarily negative, but only because the operator advanced the obstacle onto a stationary tool, which would credit a policy with an incursion it did not command. And only the AEGIS condition applies the epistemic buffer (both baselines run at ${\beta }_{\mathrm{epi}}=0$) so part of the separating margin is a constant offset, though not all of it: against the same 12 mm constant the three AEGIS runs clear by $+19.7$, $+61.1$ and $+64.7$ mm, the belief term supplying $1.6$, $31.2$ and $14.9$ mm. Table 6 separates the two effects at the kinematic tier, where the buffer is held common.

Three qualifications belong with these numbers. First, n is small and unequal: the negative clearance occurred in one run of six, so the defensible claim is that trust-latest admits an incursion AEGIS does not. Symmetrically, no incursion in three AEGIS runs bounds nothing (the rule-of-three bound is vacuous at $n=3$) so what these runs establish is a paired observation together with the mechanism of Figure 7 that explains it. Second, AEGIS delivered in two of three runs; the third ended in a protective stop because the box lay inside the destination’s keep-out margin and the endpoint guard refused to release onto it. Third, clearance is measured against the detected footprint rather than observed contact, and depth was unavailable in this session, so sensing ran on colour alone.

5.3. Latency Budget of the Layered Architecture: Which Layer Can Own Which Hazard

Recent analyses of vision–language safety mechanisms identify a safety–latency trade-off as a central open problem: the time a semantic monitor spends reasoning can itself admit the collision it is meant to prevent [22]. Every reaction time here is measured (the geometric projection costs $0.037$ ms per call (95th percentile $0.043$ ms) against a 40 ms simulated command cycle (25 Hz), and the semantic chain is timed end to end on the arm below) so the division of labour between the layers can be derived rather than asserted.

We measure the chain directly on the physical NED3 Pro. The arm executes a slow sweep while the monitor issues verdicts continuously on the live camera stream; an independent, computationally cheap skin-colour detector running at 30 Hz provides the ground-truth instant at which a human hand crosses into the workspace, and the arm is declared frozen when its measured joint motion falls below 2 mrad. Hand entry, verdict arrival and joint state are stamped by a single monotonic clock inside one process, so the intervals in Table 8 are differences on one timebase and carry no inter-device synchronisation error. This campaign was run on the hosted backend, so its $1.21$ s raw query latency is the pessimistic end of Table 4 and the resulting budget is a worst case; the three verdict latencies quoted in this paper are not in conflict but measure three different conditions: $0.24$ s for the on-device model benchmarked in isolation, $0.39$ s for the same model contending with the control loop during the AEGIS hardware session (Section 5.2.3.1), and $1.21$–$1.6$ s for a hosted backend over the network. Over $n=5$ intrusions and 109 verdicts (Table 8), the interval from hand entry to the semantic veto is $1.29$ s on average (worst $1.95$ s), the in-flight command then takes a further $0.44$ s to complete before the arm is still (worst $0.72$ s), and the total hand-to-freeze reaction is $1.73$ s on average with a worst observed value of $2.67$ s. The actuation term dominates the geometric layer’s budget: its decision compute is four orders of magnitude smaller than the $0.44$ s it must wait for the arm’s blocking command interface, so $T_{\mathrm{react}}\approx 0.44$ s for the geometric layer on hardware and $1.73$–$2.67$ s for the semantic one.

For an assumed inward-speed bound v and a conservative end-to-end reaction bound $T_{\mathrm{react}}$, the reaction allowance in Equation (6) is $v{T}_{\mathrm{react}}$. The values measured here are observed timings rather than certified bounds; they can support design estimates but cannot guarantee interception at a specified hazard speed. The measurements support keeping geometric checks active for moving intrusions and using the VLM to modulate the response to sufficiently persistent semantic conditions.

A further observation sharpens what the semantic layer is for. Presenting the monitor with a large, unfamiliar black equipment box intruding into the workspace returns safe, under both the hand-specific prompt and a stricter one that explicitly forbids any foreign object; the same monitor returns abort on a human hand within $1.0$ s. The semantic layer is therefore not a motion or novelty detector that happens to be slow; it discriminates hazard semantics, which is precisely the capability the geometric layer lacks and precisely why an inert intruder is left to the geometric clearance constraint rather than to a veto.

5.4. Simulation Evidence

The simulation campaign is reported in the Supplementary Material rather than here. It covers the accumulated-cost and intervention audit of the projection layer, safe exploration under injected action noise, a control-barrier-function baseline, the matched comparison of the scripted replanner against the learned ACT policy behind the identical layer, and the use of the simulator as a pre-deployment safety screen. Its role in this paper is bounded: it establishes that the layer is policy-agnostic and minimally invasive at a sample size hardware cannot reach, while every claim about the sensing chain and about AEGIS on the physical arm is established here.

6. Discussion

The central result of this study is architectural rather than numerical: separating the task policy from the geometric layer produced no reported grasp-site obstacle contacts for the tested classical and learned policies, while modifying only a small fraction of commands in the simulation audit. This is the reason the same layer wraps an RRT planner and an action-chunking transformer identically, and the reason the learned policy still needs it: a network that is safe on average still emits occasional out-of-distribution commands, and per-step constraint checking can reject such commands under the modeled geometry.

The dynamic-obstacle experiments clarify when explicit replanning and reactive learning are respectively preferable. At fixed obstacle height either is adequate (both exceed $95\%$ at zero collisions); at varied height the planner’s band-restricted search often fails to find a route the feasibility oracle shows exists, so it stops safely but completes far less, whereas the reactive learned policy flows around many of the same cases. The practical reading is that a reactive learned controller behind the same hard layer extends the completable envelope beyond our sampling-based replanner while retaining the implemented grasp-site constraints, not that learned control is superior in general, which the benchmark makes directly testable.

The feasibility oracle is what makes that reading defensible rather than anecdotal. Since none of the learned policy’s missed episodes were infeasible, its ceiling is a property of the training procedure (one round of self-imitation helps, a second regresses as cloning one’s own imperfect successes compounds bias, and added capacity does not help); and since only $3/150$ scenarios were infeasible overall, the planner’s much larger shortfall is its own search conservatism. Reporting this decomposition (infeasible versus planner-conservative versus imitation-missed) identifies which improvements would raise completion and which would not.

Scope and Limitations

The evidence here is of two kinds and the boundary between them is stated precisely. The safety and sensing chain (the geometric projection, the reactive detour, camera-based intruder detection with RGB-D confirmation, and the on-device semantic monitor) is validated closed-loop on the physical NED3 Pro. The policy comparison and the large-sample statistical campaign are simulation-scoped by design. Six qualifications bound what the results establish; the directions they open follow.

(i) Simulation-scoped statistics: The large-N collision, completion and AUC numbers come from the simulator and a domain-randomised surrogate, so the statistical sim-to-real claim is predictive. The hardware evidence is a set of demonstrative closed-loop trials (tens of carries with routing, freezing, live moving obstacles and a $30/30$ autonomous run) not a matched large-N replication. The simulated carry also latches a firm grasp rather than modelling in-hand slip, and its sudden obstacle is static once it appears.

(ii) Small n at the hardware tier, against reference policies: The hardware sample is small and unequal ($6/3/2$ runs); the single negative clearance is attributable to a four-tick detection dropout rather than to the verdict rule, and dropouts of that kind occur in every run of the session, so the supported claim is that trust-latest’s smaller margin admits an incursion AEGIS’s did not. The protective distances were scaled to the platform ($R_{\mathrm{human}}=6$ cm against 10 cm in simulation) because a $22.4$ cm carry cannot admit the simulated bubble; the ordering is preserved but absolute clearances are not comparable across tiers. The two comparison policies are deliberately degenerate endpoints (Table 3), not competitive baselines from the delay-aware or robust safety-filter literature; the buffer-matched comparison that isolates the rule itself is Table 6.

(iii) The propositions are conditional: They are stated for the constrained tool point rather than the swept volume and depend on Assumptions A1 and A2. The semantic-score lower bound is not established by the pilot data (Remark 1), so the buffer condition remains a design obligation rather than a verified property of the deployed constants.

(iv) Clearance is enforced at the grasp site: We audited the links the constraint does not cover rather than assume they follow: over 120 episodes the forearm crossed the hazard sphere in $5/60$ (ACT) and $2/60$ (planner). The guarantee is therefore exact for the constrained point and quantified for the rest, and a whole-arm extension is the prerequisite for human-adjacent deployment.

(v) What the sensing chain can and cannot see: One fixed camera and one calibrated plane: the homography is valid only on the table and the arm can occlude the hazard it is reasoning about. The intruder detector’s three workspace-gating rules each define a way for a person to go unseen (a background reference captured at the start of the carry, rejection of blobs connected to the top of the frame, and a centroid-in-workspace test) and the semantic monitor was validated on a small single-scene, single-subject set, so reliability across subjects, viewpoints, lighting and occlusion is uncharacterised.

(vi) The evidence clock: Verdict age is measured from inference completion rather than image capture, so the enforced radius is under-sized by up to $29.8$ mm in simulation and $17.9$ mm on the arm; the epistemic buffer covers most but not all of it at either tier (Section 4).

7. Conclusions

A semantic verdict becomes stale after the source image is captured, and a safety margin that ignores this age is either permanently conservative or can remain too small during a blind interval. This paper introduced AEGIS, a safe-action projection whose hold radius depends on a semantic belief and on the age of the corresponding evidence, and derived a conditional grasp-site separation result together with a fresh-object path-feasibility property. The distance decomposition is informed by speed-and-separation concepts, but the prototype is not an implementation or certification of ISO/TS 15066. AEGIS was examined at three evidence tiers, and the result is a position in a trade-off rather than dominance over either endpoint. Where the obstacle persists, always assuming a person delivers nothing at all while AEGIS delivers every episode and stays collision-free; where the intruder eventually leaves, holding costs nothing and that same endpoint is the safest rule available. AEGIS pays neither price in full, and on the safety axis it sits nearer the policy that trusts the latest verdict than the one that ignores it. Against that policy (the informative comparison, since both deliver) a buffer-matched kinematic campaign separates the rule from the constant margin and finds the age-indexed rule significantly safer, $41\%$ of episodes free of any commanded approach to a person against $27\%$ ($n=250$ matched seeds, $p=2.7\times {10}^{-4}$), with no constant belief competitive on both axes at once (Table 6). The physical tier does not hold the buffer fixed and is reported as a mechanism demonstration rather than as the primary comparison.

That mechanism rests on a sensing chain closed on real hardware: planar image-to-base mapping with millimetre-scale calibration residuals in the tested workspace in the robot base frame, intruder detection fused with metric depth from the same camera, and an on-device semantic verdict in $0.24$ s, an order of magnitude faster than hosted backends, and the reason the verdict can gate a carry at all. With that chain the identical policy produced opposite behaviours under the two verdicts, routing around an inert obstacle and freezing for a human hand before resuming and delivering, while completing $30/30$ autonomous carries. The observed reaction times support keeping the geometric layer active for moving intrusions and using the slower semantic layer to modulate the response to sufficiently persistent scene conditions. Beneath both, the geometric projection reduced the reported accumulated constraint cost in simulation while raising task success, and wrapped a classical planner and a learned action-chunking transformer without retraining. The main conclusion is that step-wise geometric admissibility, calibrated camera sensing, and semantic monitoring are best treated as first-class policy-agnostic elements coupled by a margin that knows how confident and how recent its evidence is. The clearest next steps follow from the stated limitations: a matched large-sample hardware campaign to replace the present demonstrative trials, a competitive delay-aware baseline in place of the two degenerate reference policies, and a multi-subject, multi-viewpoint characterisation of the semantic monitor.