Towards Human-Centered Safety in Vision-Language-Action Robot Control: A Survey and Outlook

1. Introduction

Robotic systems are increasingly entering human environments, enabling applications that range from autonomous vehicles [1,2], delivery robots [3,4], drones [5], and underwater platforms [6], to manipulation agents [7,8] and emerging humanoids [9,10]. Recent advances in VLA models further accelerate this transition by connecting visual perception and language understanding to physical action in different architectural forms, from monolithic policies that directly predict control [11], hierarchical systems with intermediate reasoning [12], and agent-based systems that generate programs or call skills [13]. World Action Models (WAMs), despite specialized coinage, have also followed these architectural patterns, from non-reasoning, monolithic DreamZero [10] to skill-calling, agentic AW-0 [14]. These architectures retain a common link from perception and language to physical action, so errors in interpreting observations or instructions can have serious physical consequences. For example, an unsafe household robot asked to “clean the kitchen” may poorly place a knife near a child, or discard a personally valuable object. We therefore refer to VLA control broadly to encompass existing systems for how safety risks may emerge and examine how safety-relevant understanding must constrain their actions.

As robots can operate closely around human, the growing importance of their safety is especially highlighted by the “Statement on AI Risk” [15] and recognized as a global priority. Yet, for VLA robot systems, safety depends on contextual factors that extend beyond geometric feasibility, involving human vulnerability, the value and ownership of objects, environmental conditions, and social expectations in human-shared environments that are difficult to enumerate [16]. This also reflects a long-standing need in robotics that has yet been realized, as expressed through Asimov’s First Law [17,18], [ colback=lawone, colframe=lawone, boxrule=0pt, arc=2pt, left=6pt, right=6pt, top=5pt, bottom=5pt ] “A robot may not injure a human being or, through inaction, allow a human being to come to harm.” —- Isaac Asimov In fact, translating this high-level principle into general robot behavior has been unresolved, as robots must infer harm from context, reconcile competing human, environmental, and social constraints, and determine how these constraints govern physical action while relying on incomplete human specifications [19]. Contemporary robot-safety frameworks exhibit similar limitations, as recent standardization efforts note that existing frameworks do not fully address robots that integrate mobility, manipulation, dynamic balance, and close human interaction [20]. Standards such as ISO 10218 for industrial robots, ISO 13482 for personal-care robots, ISO/TS 15066 for collaborative industrial robot systems, IEEE 7001 for transparency in autonomous systems, and ISO/IEC 24029 for neural-network robustness provide important foundations, but these frameworks primarily address physical safety, transparency, or robustness through control-level properties such as stability [21,22] and collision avoidance [23,24]. Conversely, model-level AI safety has largely focused on semantic and behavioral harms in digital outputs [25], where model predictions are not directly coupled to physical execution.

In essence, VLA systems connect two previously separated safety problems: whether an action is semantically appropriate in context and how to causally ensure that this understanding constrains the physical control.

1.1. Human-Centered Semantic Action Safety

VLA systems sit at the intersection of two safety traditions, inheriting semantic reasoning failures from agentic models while producing embodied actions that directly affect humans, objects, and shared physical environments. This intersection motivates the problem of semantic action safety [12], where safe physical behavior depends on understanding context-dependent constraints that cannot be specified solely through geometric or control-level rules. From a human-centered perspective, these constraints concern how robot behavior affects human well-being, environmental integrity, and social compliance, as we ground this perspective in IEEE’s Ethically Aligned Design, which calls for autonomous and intelligent systems to “remain human-centric, serving humanity’s values and ethical principles” [26]. For VLA systems, this principle requires human-centered values to be translated from semantic understanding into physical behavior. Accordingly, we use human-centered semantic action safety to describe the capability of a VLA system to recognize safety-relevant context and generate actions that remain compatible with these human-centered constraints. The central question is therefore whether safety-relevant semantics are sufficiently represented by the model in alignment with human values, and whether they remain consequential for the physical action.

Figure 1. Overview of human-centered safety in VLA robots. The figure illustrates the proposed lifecycle-oriented taxonomy for semantic action safety in VLA systems, spanning design-time, deployment-time, and validation-time risks. It highlights how VLA policies tightly couple multimodal perception, semantic reasoning, and embodied control, requiring safeguards that address human safety, environmental integrity, and social compliance throughout development.

Figure 1. Overview of human-centered safety in VLA robots. The figure illustrates the proposed lifecycle-oriented taxonomy for semantic action safety in VLA systems, spanning design-time, deployment-time, and validation-time risks. It highlights how VLA policies tightly couple multimodal perception, semantic reasoning, and embodied control, requiring safeguards that address human safety, environmental integrity, and social compliance throughout development.

(a) What is semantic action safety for VLA robots?

Although action safety has long been central to robotics through specifications, planning, control guarantees, and physical safeguards [21,22,23,24], VLA systems introduce an additional safety challenge because semantic representations can directly shape physical action [7,8,16,27]. From a human-centered perspective, this includes semantic constraints arising from human intent and well-being, object and environmental context, and social expectations [12,16]. In VLA systems, vision and language provide information about goals, constraints, and context, while generated actions determine real-world outcomes. Errors in interpreting instructions, affordances, preferences, social context, or environmental constraints can therefore propagate into unsafe behavior [28].

Semantic information may enter the VLA pipeline through different representations rather than appearing only at the final action output. It may remain implicit in multimodal latent features learned under safety-critical objectives [29], or become explicit through intermediate reasoning states, chain-of-thought traces, plans, or predicted consequences [7,8]. Accordingly, we use “reasoning” in an operational rather than human-cognitive sense [30], referring to intermediate representations that support action-relevant inference during policy execution. We therefore question whether these representations merely encode safety-relevant semantics or whether they form semantic-to-action causality, where semantics remain consequential for the physical action ultimately generated.

(b) Why is semantic action safety uniquely challenging? For robots acting in open-ended human environments, semantic action safety remains challenging for two major reasons: the relevant safety constraints are difficult to specify exhaustively, and their influence must be preserved from semantic understanding through physical execution. First, safe behavior cannot be determined from physical feasibility alone. Although the space of possible safety cases is too broad to enumerate exhaustively, recurring concerns in human-robot safety [16], standardization [20], and foundation-model safety [19,25] can be organized around three human-centered dimensions:

• Human well-being, which concerns risks to health, bodily integrity, and psychological comfort.
• Environmental integrity, which concerns damage to shared spaces, infrastructure, and human-valued objects.
• Social compliance, which concerns violations of rules, norms, preferences, and values.

Semantic action safety therefore raises an objective-specification problem, where the admissibility of an action depends on context-dependent human-centered constraints and on its anticipated consequences, rather than on whether it can simply be executed.

This problem reflects a broader critique of the standard model of AI, where models optimize fixed objectives even though “we have no reliable way to make sure that their objectives are the same as our objectives” [19], while safety-relevant conditions remain difficult to enumerate exhaustively for general objectives [12]. For VLA robots, this problem becomes embodied and partially observable because a language instruction provides only a partial and underspecified objective, while visual observations provide only partial evidence about the people, objects, environments, and norms that constrain safe execution. Semantic action safety can therefore be viewed as an embodied form of the value-alignment problem, where useful behavior requires compatibility between interpreted task intent and human-centered safety constraints across perception, reasoning, and control.

Second, correctly recognizing a safety-relevant constraint does not ensure that it will affect the action ultimately executed. Its influence may be weakened or lost across the VLA pipeline. For example, action misalignment may originate from design-time choices in data, objectives, or modeling [31,32,33], remain hidden when validation does not cover semantic, long-tail, or human-centered cases [4,12,33], and surface during deployment under uncertainty, out-of-distribution observations, noise, visual perturbations, or adversarial inputs [34,35,36,37]. The challenge is therefore lifecycle-wide, as safety-relevant distinctions must be captured during learning, preserved through action generation, enforced when execution becomes unsafe or uncertain, and evidenced under safety-critical evaluation.

We therefore organize existing approaches according to where they operationalize human-centered safety across the VLA lifecycle:

• Design time. Data, objectives, and modeling choices that determine which safety-relevant distinctions are learned and how they shape action generation.
• Deployment time. Monitoring, steering, recovery, and human escalation mechanisms that determine how safety judgments alter behavior during execution.
• Validation time. Benchmarks, stress tests, and safety evidence that assess whether safety-relevant conditions produce the required behavioral changes before unsafe outcomes occur.

Recent industrial safety efforts adopt related lifecycle-oriented views, with NVIDIA Halos exemplifying a full-stack AI safety architecture spanning design, deployment, and validation from autonomous driving to broader robotics [38,39]. Importantly, these categories describe where a safety mechanism acts rather than when it is engineered; deployment-time safeguards, for example, must still be specified, integrated, and validated before release even though they intervene during execution. The lifecycle therefore identifies where semantic action safety is operationalized, while our later synthesis examines what must be captured, preserved, enforced, and evidenced for safety-relevant semantics to remain consequential for physical action.

1.2. Scope and Contributions

1.2.1. Bridging AI Safety and Robotic Safety

AI safety and robotic safety have historically emphasized different parts of the safety problem. Research on multimodal and foundation-model safety has largely studied alignment, adversarial robustness, harmful outputs, and failures in semantic reasoning [25,40], while robotic safety has developed methods for physical interaction, planning, control guarantees, runtime assurance, and human-robot safety [16,22]. VLA systems increasingly connect these concerns because semantic interpretation is directly coupled to physical action. A failure may originate from an instruction, multimodal representation, intermediate reasoning state, or predicted consequence, yet become safety-critical only when it changes the action executed by the robot. Recent work on formally guaranteed AI safety [41], human-AI safety [16], and VLA safety has begun connecting semantic reasoning with physical constraints and runtime intervention. However, it remains unclear how this interface preserves safety-relevant information and uses it to guide robot actions. We refer to this relationship as the central problem underlying human-centered semantic action safety.

1.2.2. Systematic Scope and Objectives

This survey focuses on human-centered semantic action safety in VLA systems operating in open-world physical environments, examining how visual observations and language instructions shape physical actions through implicit or explicit semantic representations. Under our broad definition of VLA control, this includes monolithic [11,29], hierarchical [12,42], and agent-based systems [13]. World-action modeling is orthogonal to this architectural distinction, spanning monolithic implementations such as DreamZero [10] and agentic AW-0 [14].

Earlier surveys provide important foundations but approach the problem from different levels of abstraction. General surveys of embodied AI and VLA models primarily organize progress around architectures, multimodal representations, task planning, control policies, datasets, and downstream capabilities [43,44]. Surveys of autonomous-driving safety instead study perception, prediction, planning, and control under driving-specific safety assumptions [45]. Broader AI and machine-learning safety surveys characterize alignment, robustness, uncertainty, and model-level failure modes [25,40], while security-oriented embodied-AI surveys emphasize adversarial vulnerabilities, attack surfaces, and corresponding defenses across embodied perception, reasoning, and action [46]. These perspectives establish important foundations for understanding VLA safety, but differ in whether their primary object of analysis is model architecture, domain-specific control, general AI safety, or adversarial robustness.

Recent surveys have begun addressing VLA and embodied-AI safety from several complementary perspectives. Yuan et al. [47] organize VLA safety across data preparation, model training, and deployment, providing broad lifecycle coverage but placing less emphasis on how safety-relevant semantics remain behaviorally consequential across these stages. Li et al. [48] focus primarily on training- and inference-time threats and defenses, which gives strong coverage of adversarial safety but less directly addresses non-adversarial semantic misalignment and human-centered action consequences. Broader embodied-AI surveys cover safety across perception, reasoning, planning, action, and interaction [49], but their wide system scope limits detailed analysis of how VLA-specific semantic representations influence physical control. Our survey complements these works by centering on human-centered semantic action safety and examining when safety-relevant semantics become consequential for physical action across design, deployment, and validation of VLA models.

1.2.3. Survey Contributions

Our survey studies VLA safety through a human-centered semantic action perspective, focusing on how safety-relevant semantics become consequential for physical behavior. We make the following contributions.

• Human-centered semantic action safety. We characterize semantic action safety around context-dependent constraints on human well-being, environmental integrity, and social compliance, with particular attention to how semantic understanding influences physical action.
• Empirical lifecycle analysis. We systematically organize and analyze existing work across design time, deployment time, and validation time, examining how safety-relevant information is learned, acted upon during execution, and evaluated under safety-critical conditions.
• Semantic-to-action synthesis. We identify four linked requirements for semantic-to-action safety, including safety boundary coverage, semantic-to-action fidelity, contextual intervention and recovery, and incident-path validation.
• Open challenges and research agenda. We connect these requirements to recurring gaps in current systems, including sparse boundary examples, limited causal verifiability, gaps in intervention timing and recovery, and incomplete incident attribution.

Together, these contributions provide a structured account of how safety-relevant semantics can remain consequential for physical action across the VLA safety lifecycle.

2. Review Methodology and Empirical Landscape

We conduct a structured literature review to characterize how human-centered safety information is represented, operationalized, and evaluated in VLA robot systems. Because this literature is rapidly evolving across publications, preprints, benchmarks, technical reports, and project pages, we construct the corpus through broad literature search and web crawling, followed by expert-guided supplementation, backward and forward citation tracing, screening, and structured coding.

2.1. Review Methodology

Search objectives. Our review is guided by three questions. (i) How is human-centered safety information represented, and how does it influence physical action in VLA systems? (ii) How do design-time, deployment-time, and validation-time mechanisms capture, preserve, enforce, and evaluate this relationship? (iii) What technical assumptions, trade-offs, and recurring gaps emerge across these approaches?

Corpus construction. We first construct a broad candidate corpus using manually defined queries covering VLA systems, embodied AI, robot safety, autonomous systems, human-robot interaction, and related safety concepts across Google Scholar, Semantic Scholar, arXiv, and broader web search. Expert-guided search is then used to recover relevant terminology, recent work, and adjacent research communities that may not be retrieved reliably through fixed queries. We further apply backward and forward citation tracing from retained papers. Records corresponding to the same work across preprint, conference, and journal versions are consolidated, retaining the most complete or latest archival version. The complete search queries, search dates, inclusion and exclusion criteria, and coded corpus are provided in the supplementary material and public corpus spreadsheet in our GitHub repository.

Screening, coding, and verification. The resulting 1352 papers are screened at the abstract level with GPT-5.6 Sol Pro to support VLA- and safety-relevance filtering, yielding 404 VLA-relevant works. We subsequently use full-text analysis to support evidence extraction and coding of safety mechanisms, lifecycle stages, protected values, embodiments, evaluation scenarios, and recurring trends, resulting in 286 papers with explicit safety evaluations. The taxonomy, coding vocabulary, and inclusion criteria are manually defined, while all retained evidence, classifications, and trend assignments are manually verified against the source papers. Comparative analysis, interpretation, and technical synthesis are conducted manually by the authors.

In particular, we separately mark an evaluation as human-involved when a human is an explicit actor in the evaluated interaction and the robot’s safety behavior depends on, responds to, or affects that interaction. This category contains 123 papers and includes both real and simulated human interactions, so it does not imply that a physical human participant is present. Human involvement is distinct from the protected value of human well-being, which may be evaluated without an interacting human. Lifecycle labels are non-exclusive when a paper contains distinct safety mechanisms operating at multiple stages, e.g., [29], which contributes both design-time alignment and validation-time benchmarking.

2.2. Empirical Landscape of VLA Safety Research

Figure 2 summarizes the reviewed corpus. Among 1352 robot-model papers mentioning safety, 404 are VLA-relevant, 286 report explicit safety evaluations, and 123 include human-involved evaluation.

Human-centered evidence remains limited. Human-involved evaluations account for approximately 43% of safety-evaluated papers. Most evaluations therefore do not include a human actor whose state or behavior directly affects the robot’s safety response. Because this category also includes simulated interactions, direct physical evaluation with people represents an even smaller subset. This motivates stronger evaluation of safety as an interaction-dependent property rather than only a property of task completion or robot motion.

Safety extends beyond physical harm. Protected-value labels are non-exclusive, with 119 papers evaluating human well-being, 237 environmental integrity, and 139 social compliance among the 286 safety-evaluated papers. Figure 2(d–f) further shows substantial diversity within each dimension, ranging from physical hazards to object damage, contextual constraints, and socially inappropriate behavior. These patterns support a human-centered view in which safe action depends on people, environmental context, and social constraints rather than physical feasibility alone, while exposing the difficulty in trying to exhaustively enumerate.

Deployment-time safety receives substantial attention. Among safety-evaluated papers, 93 contribute design-time mechanisms, 149 deployment-time mechanisms, and 124 validation-time mechanisms, with lifecycle labels treated as non-exclusive. The prevalence of deployment-time mechanisms reflects substantial attention to runtime monitoring, filtering, verification, steering, and intervention. This suggests that design-time alignment alone is generally not treated as sufficient for safe deployment.

Implications for semantic action safety. These observations motivate examining whether safety-relevant information remains consequential for physical action across the lifecycle. We therefore study how such information is learned at design time, how it constrains or redirects behavior during deployment, and how its influence is assessed under safety-critical validation.

Historical progression of safety mechanisms.Figure 3 places these empirical patterns in a broader technical progression. Design-time work has moved from general-purpose robot and driving data toward safety-critical and long-tail data, while semantic-to-control methods increasingly make action consequences and safety reasoning explicit. Deployment-time methods have progressed from formal reachability and barrier-based filtering toward semantic anomaly detection, uncertainty-aware intervention, and contextual recovery. Validation has similarly expanded from fixed closed-loop benchmarks toward long-tail, human-centered, and generative stress testing. These trends motivate the following lifecycle analysis, where we examine what safety information is learned, how it influences action, and how that influence is exposed and verified.

In this paper, our lifecycle analysis discusses 49 works on design-time safety, 44 on deployment-time safety, and 57 on validation-time safety, comprising 133 distinct works after accounting for overlap across stages. The bibliography contains 179 references overall, including supporting sources cited elsewhere. Among the surveyed works, we highlight milestones in autonomous driving and robotic manipulation that, in our assessment, represent breakthroughs in safety-relevant capabilities.

3. Semantic Action Safety Problem Formulation

Following [50], we adopt a minimal formulation connecting semantic representation, action generation, and safety consequences, introducing only the notation used in the remainder of the survey.

3.1. Semantic-to-Action Interface

Let s denote the environment state, o the robot observation or observation history, ℓ the language instruction, and a the action, which may be a low-level command, an action chunk, or a skill call. The state s includes safety-relevant attributes that o may capture only partially, such as object temperature or fragility, ownership, the presence and intent of nearby people, and applicable norms. A VLA policy generates

$$a\sim {\pi }_{\theta }(·\mid o,\ell )$$

(1)

through an intermediate semantic representation r, e.g., that an object is a sharp knife close to the edge. Monolithic VLAs encode r implicitly [11,51,52], whereas hierarchical, agent-based, and world-action models expose it as reasoning, plans, skill calls, or predicted futures [10,12,13,27,42,53,54,55,56,57,58,59] (Figure 4). Here, r denotes the semantic information represented by the model, which may be incomplete or inaccurate with respect to the safety-relevant attributes of s. We therefore distinguish failures to represent safety-relevant information from failures to preserve that information in action. When safety-relevant information is correctly represented in r, safe behavior further requires that it appropriately shapes a. We call behavioral evidence that such distinctions are preserved in action semantic-to-action fidelity, and reserve semantic-to-action causality for the stronger claim that intervening on a represented distinction changes the action.

3.2. Human-Centered Semantic Action Safety

Human–AI safety and robot-safety work emphasize that safety depends on human and operational context as well as physical feasibility [16,20]. A safe VLA action therefore accounts for human vulnerability, object value and ownership, social expectations, and delayed consequences, in addition to being collision-free and task-consistent.

We define safety through its consequences. An incident is a realized harm or violation of a protected value under the applicable contextual safety requirements. Task-required or consented changes, such as cutting a vegetable or contact during a handover, are not incidents when they remain within those requirements. We classify incidents by what bears the harm, following the three protected values of human well-being, environmental integrity, and social compliance:

$${\mathcal{S}}_{\mathrm{inc}}={\mathcal{S}}_{\mathrm{hum}}\cup {\mathcal{S}}_{\mathrm{env}}\cup {\mathcal{S}}_{\mathrm{soc}},$$

(2)

where ${\mathcal{S}}_i$ collects the states in which an incident of type i has occurred, and the type is given by what the incident affects:

• ${\mathcal{S}}_{\mathrm{hum}}$: a person’s body, e.g., a cut, a burn, a strike, or contact force above the permitted limit [60].
• ${\mathcal{S}}_{\mathrm{env}}$: a non-human physical entity, e.g., an object, surface, or food that is damaged, contaminated, consumed, or lost.
• ${\mathcal{S}}_{\mathrm{soc}}$: a rule, permission, privacy expectation, or social norm that has been violated, e.g., entering a restricted area or using another person’s belongings without permission.

These categories need not be mutually exclusive: an incident may affect multiple protected values, and a state belongs to every corresponding incident set. For example, damaging another person’s belongings without permission may involve both physical damage and a permission violation. An incident is realized once the change has occurred, even when its consequences surface later, as with contaminated food. What the instruction and the affected people permit is set by the deployment context, its stakeholders, and applicable standards, and forms the normative input of the formulation.

Because outcomes are stochastic and many useful tasks carry inherent risk, we follow the standard view that safety means keeping risk within a tolerable level [61]. Numeric tolerances are specified for physical cases such as human–robot contact forces [60]. To compare incident probabilities across all three protected values, we adopt a careful and competent human policy, denoted ${\pi }_{\mathrm{ref}}$, as a normative reference for responding to the same instruction in the same situation. This reference is assumed to satisfy the applicable contextual safety requirements, including stopping or declining the instruction when necessary. We define the resulting reference-based safety margin as:

$$\begin{array}{cc}\hfill F_{\mathrm{safe}}(s,a)=\underset{i}{\min }(& {Pr}_{{\pi }_{\mathrm{ref}}}\left[\mathrm{reach}{\mathcal{S}}_i\mid s\right]\hfill \\ \hfill & -{Pr}_{{\pi }_{\theta }}\left[\mathrm{reach}{\mathcal{S}}_i\mid s,a\right]),\hfill \end{array}$$

(3)

where $i\in \{\mathrm{hum},\mathrm{env},\mathrm{soc}\}$, “reach ${\mathcal{S}}_i$” denotes entering ${\mathcal{S}}_i$ before the episode ends, the robot executes a and then follows ${\pi }_{\theta }$, and the reference human acts from the same state, including stopping or declining the instruction. In practice, Equation (3) defines a safety target relative to a chosen normative baseline, which may be unavailable or subject to differing human judgments, while incident probabilities under either policy may be difficult to estimate. Nevertheless, existing works have leveraged demonstrations, safety specifications, intervention feedback, and incident observations to provide partial evidence relevant to this target depending on their representativeness of scenarios.

An action satisfies this reference-based safety criterion if $F_{\mathrm{safe}}(s,a)\ge 0$, meaning that no incident category is more likely than under the reference policy, while a negative value of $F_{\mathrm{safe}}(s,a)$ indicates that this requirement is violated for at least one category. The minimum applies this comparison to each incident type separately, so the risk of injuring a person is judged on its own, independently of the risk of breaking a cup. The comparison is context-dependent through the reference policy, including its ability to stop or decline an unsafe instruction. However, the criterion compares incident probabilities within each category and does not distinguish different severities of harm within that category. Semantic action safety thus acts as a constraint within which the policy pursues useful task behavior.

3.3. Lifecycle Operationalization

Equation (3) defines the target that a VLA system approximates: it depends on the true state s, on future outcomes, on a hypothetical reference human, and on a partially specified ${\mathcal{S}}_{\mathrm{inc}}$. Existing approaches operationalize it through different forms of evidence, proxies, and intervention across the lifecycle.

Design-time safety. Safety enters learning through safety-relevant evidence ${\mathcal{D}}_{\mathrm{safe}}$ and the training objective that shapes how ${\pi }_{\theta }$ maps o through semantic representation r to action a. Human demonstrations can be viewed as samples of ${\pi }_{\mathrm{ref}}$ and mostly cover states well inside the safe region, since incidents are dangerous and expensive to collect, with states near $F_{\mathrm{safe}}(s,a)=0$ especially calling for dedicated data. Imitation-driven approaches encode safety through demonstrations, constrained methods introduce explicit safety costs [29], and reasoning- and world-model-based methods expose semantic or consequence representations before control [27,42,53]. Design-time safety therefore asks whether training captures the contextual distinctions that ensure the positive side of $F_{\mathrm{safe}}$ and preserves them through r to action.

Deployment-time safety. At runtime the system observes o, a partial view of s, so safeguards construct an estimate

$${\mathcal{C}}_{\mathrm{safe}}(o,\ell ,a)\approx F_{\mathrm{safe}}(s,a),$$

(4)

whose accuracy depends on outcome stochasticity, on how well learned consequence models generalize under distribution shift, and on how visible hazards such as heat or intent are in o. Reachability and latent-safety methods [34,62,63], CBF-based filters [64,65], predictive safeguards [66,67,68], and guided generation [69,70,71,72] instantiate ${\mathcal{C}}_{\mathrm{safe}}$ and use it to pass, select, modify, block, replan, or escalate to a human. Deployment-time safety therefore asks whether a runtime judgment is sufficiently grounded to produce a context-appropriate behavioral change while safe recovery is still possible.

Validation-time safety. Validation estimates whether $F_{\mathrm{safe}}(s,a)\ge 0$ holds across deployment-relevant conditions from finite tests, including closed-loop and long-tail benchmarks [33], manipulation benchmarks that measure collision and damage [73,74], and generative stress testing [36,75]. Its evidence is strongest when scenarios cover the conditions leading to ${\mathcal{S}}_{\mathrm{inc}}$, test dynamics preserve the interactions that produce incidents, and incident detectors and human judgments closely approximate ${\mathcal{S}}_{\mathrm{inc}}$ and ${\pi }_{\mathrm{ref}}$. Finite validation therefore provides evidence for the tested conditions, which strengthens as coverage widens.

Under this formulation, the lifecycle provides a way to trace where safety-relevant information is introduced, acted upon, and evaluated. The following sections examine whether this information is captured through safety boundary coverage, preserved through semantic-to-action fidelity, enforced through contextual intervention and recovery, and evidenced through incident-path validation.

4. Design-Time Safety

Design-time methods differ in how they operationalize the unknown $F_{\mathrm{safe}}$ through safety-relevant evidence ${\mathcal{D}}_{\mathrm{safe}}$, the training objective, and the learned grounding from o through r to action. We therefore examine not only what safety signal a method introduces, but whether that signal is sufficiently informative, appropriately valued during optimization, and consequential for the action ultimately generated. Ideally, semantic action safety emerges when ${\pi }_{\theta }$ learns behaviors within boundaries defined by safety-critical data and/or explicit safety constraints $F_{\mathrm{safe}}$, such that the policy can correctly recognize unsafe situations and map high-level semantic intent into control-safe realization.

4.1. Learning-Based Alignment

Following the semantic action safety formulation in Section 3.2, learning-based alignment aims to ensure that a VLA policy ${\pi }_{\theta }$ generates actions that are both semantically aligned with human values and physically safe. Existing approaches mainly differ in where safety is imposed within the policy pipeline and how the safety constraint set $F_{\mathrm{safe}}$ is represented. In particular, direct control alignment constrains low-level action generation, reasoning-augmented alignment introduces semantic reasoning or predictive feedback before control synthesis, and agentic alignment focuses on safe high-level decision-making in LLM-based agents. Together, as indicated in Table 1, these approaches reflect a shift from reactive collision avoidance toward aligning reasoning, intent, and execution with human-centered safety at scale.

4.1.1. Direct Control Alignment

Direct control alignment constrains low-level action generation to satisfy safety requirements. Although collapsing reasoning and control into implicit representations enables fast and reactive behavior, these models often fail to capture important safety semantics. Early analyses such as Multi-Hug RL [79] and S2E [80] showed how embodied systems may frequently violate physical safety or human comfort constraints, while mobile manipulation models remain prone to impacts, blind-spot errors, and unsafe interactions with fragile objects [29].

Imitation-driven safety. These methods [76,77,78] make safety learnable by training policies on trajectories that are assumed to satisfy $F_{\mathrm{safe}}(s,a)\ge 0$. When the demonstrations come from careful human operators, they can be viewed as samples of ${\pi }_{\mathrm{ref}}$ in Equation (3), which is why this assumption is reasonable within the demonstrated states. Rather than estimating the safety evaluator itself, they approximate safe behavior from the empirical support of demonstrations, teacher policies, or generated rollouts that avoid collisions and task failures. Their practical strength is that they scale naturally with heterogeneous robot datasets while preserving the reactiveness of direct control, allowing policies to be trained end-to-end and to adjust quickly when observations change due to robot motion, human motion, or background dynamics.

Within this family, recent methods differ in how much safety information the training data provides. For example, ViNT [76] shows that heterogeneous real-world navigation trajectories can induce implicit affordances such as obstacle avoidance, hallway following, road following, and pedestrian avoidance. NoMaD [77] shows that this reactive benefit depends on the action representation, since diffusion policies can preserve multiple safe action modes in pedestrian-rich and visually complex settings instead of collapsing to a single averaged response. NavDP [78] extends this data-centric shift by using simulation and privileged information to distill safer and less-safe navigation paths, giving the policy richer evidence about the local safety boundary.

It can be seen that a main technical bottleneck of imitation-driven safety is boundary density in the data, since these methods depend on how much training reveals the transition from safe behavior to the failure region where $F_{\mathrm{safe}}(s,a)<0$. Standard demonstrations usually show actions inside the safe region, but provide limited evidence about how small changes in context or action affect safety near this transition. As a result, the policy may become reactive and competent under familiar conditions while remaining unreliable in sparsely represented near-failure cases. Imitation-driven safety therefore improves average-case behavior, but its reliability depends on whether the data sufficiently covers the region around $F_{\mathrm{safe}}(s,a)=0$.

Control-constrained safety. These approaches make safety learnable by adding explicit optimization pressure against actions that are likely to cross into $F_{\mathrm{safe}}(s,a)<0$ [29,79,80]. Compared with imitation-driven safety, their strength is that they preserve the reactiveness of direct control while providing negative or corrective feedback about unsafe behavior. The central technical limitation is how to turn safety concerns into scalable reward, cost, or intervention signals that are rich enough to shape behavior, but precise enough to avoid overly conservative control.

Existing works differ mainly in how they obtain this safety signal. Multi-Hug RL [79] uses human takeovers to shape both data and reward, treating intervention as evidence that the policy is approaching unsafe behavior before collision occurs. S2E [80] scales this idea through simulation, combining offline video pretraining with RL rewards for goal reaching, collision avoidance, rule compliance, and human-like navigation. In mobile manipulation, SafeVLA [29] moves beyond generic reward penalties by introducing explicit safety costs. It uses risk-eliciting environments and a constrained Markov decision process formulation to encode hazards such as blind spots, fragile objects, critical points, and dangerous equipment.

This progression shows that control-constrained safety is not simply about adding penalties to a policy. It is about jointly shaping the training distribution and optimization objective so that safety concerns become actionable feedback for low-level control. However, the bottleneck remains proxy fidelity. Human interventions are sparse and subjective, simulation rewards cover only modeled interactions, and predicate-based costs depend on the completeness of the safety specification. As a result, these methods improve reactive safety under measurable risks, but they still do not guarantee that the learned policy will remain on the safe side of $F_{\mathrm{safe}}(s,a)\ge 0$ under unseen, long-tailed semantic hazards, rare interactive dynamics, or underspecified human-centered constraints.

Findings.Across imitation-driven and control-constrained approaches, design-time safety shifts from learning safe behavior from positive demonstrations toward learning semantic action safety boundaries that push actions away from regions where $F_{\mathrm{safe}}(s,a)<0$. Their shared strength is reactiveness, since both preserve direct action generation and adapt quickly to changing observations, while the technical trend is toward jointly shaping data and objectives through demonstration coverage, intervention feedback, and explicit safety costs. This makes the quality of the safety proxy critical, as the learned policy is safe only to the extent that the available evidence and optimization signals preserve the contextual factors that distinguish safe actions from incident-producing ones.

4.1.2. Reasoning-Augmented Control Alignment

Reasoning-augmented control alignment addresses the semantic limitations of direct end-to-end policies by introducing explicit reasoning before control synthesis. Rather than relying purely on implicit perception-to-action mappings, these approaches model objects, interactions, intent, or future outcomes to improve semantic action safety under long-tail or safety-critical scenarios. By leveraging structured reasoning and world knowledge from foundation models [95], they aim to better align high-level semantic understanding with safe low-level execution.

Reasoning-mediated safety. Rather than relying only on action likelihood or low-level reward penalties, these approaches make safety learnable by inserting semantic reasoning between perception and control, particularly reasoning over context-relevant safety factors such as critical objects, human intent, task feasibility, interaction history, temporal delay, and instruction validity to estimate whether an action should remain on the safe side of $F_{\mathrm{safe}}(s,a)\ge 0$ [4,12,81,82,83,85,86,96,97]. They can capture hazards that are not purely geometric, including semantically unsafe instructions, long-tail interactions, stale scene understanding, and context-dependent risks that reactive policies may miss.

The central technical bottleneck is causal grounding, as a model may identify a relevant object, generate a plausible rationale, or select a conservative behavior, but the reasoning is safety-relevant only if it correctly changes the downstream action. Existing work addresses this by making reasoning more tightly coupled to action generation [81,82,83,84,96]. For example, TOKEN [81] grounds planning in object-level tokens and structured question-answerings, making critical objects explicit before trajectory prediction. R&B-EnCoRe [96] treats reasoning as a latent variable and selects reasoning primitives according to their information benefit for expert action prediction. AutoVLA [82] and Poutine [83] further couple reasoning with action by training over CoT, trajectory, or vision-language-trajectory sequences, so that semantic explanations are tied more directly to physical behavior. Pinocchio [84] instead directly optimizes reasoning-action consistency with a learned faithfulness critic, while showing that improved action performance alone does not imply faithful reasoning.

A second line of work asks whether reasoning can remain useful under other dimensions of real-world control constraints [4,85,97]. For example, SpanVLA [97] reduces reasoning-to-action latency with a flow-matching action expert and uses negative and recovery samples to teach both avoidance and correction. TIC-VLA [4] shows that reasoning must also be temporally grounded, since VLM outputs may describe stale world states while the robot continues to move. StyleVLA [85] grounds semantic preferences through style-conditioned trajectory generation, where a safety-oriented mode is translated into kinematically consistent motion.

Beyond driving and navigation, Reasoning-mediated safety also appears as instruction-level and agentic gating [12,86]. For example, RationalVLA [86] introduces <ACT> and <REJ> tokens so that defective or unsafe instructions can be rejected before low-level manipulation begins; and Gemini Robotics [12] extends this idea to an agentic setting, where embodied reasoning supports task decomposition, progress estimation, tool use, safety reasoning, and recovery before executable instructions are delegated to a low-level VLA.

Together, these works show that Reasoning-mediated safety is not simply about adding explanations to VLA policies, but about making semantic risk representations structured, selective, temporally valid, and causally coupled to action generation. The main technical limitation is reasoning faithfulness, since object labels may miss hazards, CoT traces may rationalize rather than cause actions, delayed features may become stale, and instruction-level rejection does not verify the continuous safety of accepted actions.

World-model-based safety. These approaches make safety anticipatory by using predicted, reconstructed, or counterfactual outcomes to estimate whether an action may lead to violation of $F_{\mathrm{safe}}(s,a)\ge 0$ [53,87,88,89,90,91]. Rather than relying only on safe demonstrations, immediate penalties, or semantic rationales, they expose the policy, through its training objective, to what may happen after an action is taken. Their strength is that they preserve reactive control while adding future-oriented feedback, allowing the model to avoid actions that appear locally plausible but lead to collisions, traffic-rule violations, failed recovery, or unsafe interaction patterns.

The central technical bottleneck among them lies in predictive fidelity for outcome feedback, as a mechanism can be used to improve safety only when it preserves the causal factors that determine real failure, but the learning cost may still be a scalar value for optimization. Existing works have addressed this bottleneck by making action consequences progressively more informative. For example, in autonomous driving, Think2Drive [87] models what may happen through latent world-model rollouts, using imagined future states, rewards, and terminations to shape planning before execution. IRL-VLA [88] models how safe an outcome may be by learning a Reward World Model that provides denser feedback on trajectory quality, collision avoidance, rule compliance, efficiency, and comfort.

A second line of work focuses on making predicted failures and alternative futures more actionable. For example, ELF-VLA [89] explores driving failures and converts low-scoring rollouts into diagnostic feedback, so the policy can distinguish whether failure arises from reasoning, meta-action selection, safety violation, efficiency loss, or trajectory execution. TakeVLA [90] moves this signal closer to the failure boundary by using takeover and pre-takeover data, together with scenario reconstruction, to teach the policy defensive behavior before intervention is required. Curious-VLA [91] addresses the narrow-policy problem by expanding the set of feasible futures, filtering diverse candidate trajectories, and using diversity-aware RL to make safer alternatives visible during optimization. CF-VLA [53] makes consequence reasoning internal to the VLA by generating meta-actions, counterfactually evaluating its own proposed plan, and revising unsafe or suboptimal intentions before producing the final trajectory.

Together, these works show that world-model-based safety is less about predicting the future for its own sake than about making action consequences learnable for safety-aware control. However, a core limitation of predictive fidelity remains, since learned rollouts, reward models, diagnostic feedback, takeovers, generated futures, or counterfactual traces are typically used as scalar optimization objectives, and thus may fail to preserve the real causal factors needed to keep actions satisfying $F_{\mathrm{safe}}(s,a)\ge 0$.

Hybrid safety. Hybrid approaches combine Reasoning-mediated and world-model-based safety by coupling semantic hazard understanding with outcome-aware action generation [27,42]. Rather than treating reasoning as a separate explanation or world models as external evaluators, they aim to keep semantic intent, predicted consequence, and executed trajectory mutually consistent. Their strength is that they make reasoning more actionable for control while preserving some interpretability about why an action is selected.

For example, Alpamayo-R1 [42] addresses this through Chain of Causation reasoning, where observable causal factors are linked to driving decisions and then grounded in meta-actions and trajectory decoding. LCDrive [27] addresses the same issue through latent world-model reasoning, where action-proposal tokens are interleaved with latent future-state tokens so the policy can compare candidate futures before producing the final trajectory.

Together, these methods show that hybrid safety aims to close the gap between what the model recognizes as risky, what it predicts will happen, and what it physically executes. However, the limitation lies in semantic-outcome consistency, since causal traces may omit hazards, world models may hallucinate plausible but incorrect future states, trajectory rewards may miss semantic harms, and decoders may still turn plausible intermediate plans into unsafe motion [27]. Thus, hybrid safety improves reasoning-control alignment, but still does not guarantee mechanistic causality that the final action remains inside $F_{\mathrm{safe}}(s,a)\ge 0$ under rare, interactive, or underspecified hazards.

Findings.Across reasoning-augmented control alignment, the main trend is a shift from direct perception-to-action mapping toward policies that make semantic action safety explicit through structured reasoning, predicted consequences, or both. This creates a technical need for action-grounded reasoning, where semantic cues, rationales, temporal context, world-model rollouts, rewards, and counterfactual traces must not only describe risk but remain consequential for the trajectory that the VLA executes. Recent faithfulness-oriented objectives provide a behavioral means of strengthening this connection [84], but the remaining challenge is establishing whether safety-relevant intermediate representations actually govern action generation, even potentially bottlenecking performance, since explanations may remain post hoc despite behavioral consistency, predicted futures may omit or hallucinate critical consequences, and proxy rewards may fail to encode the factors that determine safe execution under dynamic open-world conditions.

4.1.3. Agentic Action Alignment

Agentic action alignment addresses the safety risks that arise when pretrained LLMs are used as high-level planners for embodied action, as unlike direct control policies, these systems often translate language reasoning into tool calls, API actions, or controller commands, so failures in instruction interpretation can render unsafe behavior. In particular, adversarial prompts and jailbreak attacks can induce unsafe physical actions when flawed reasoning is translated directly into control outputs [32]. Empirical studies have highlighted this risk. ROBOPAIR [98] achieves near-complete success in eliciting unsafe actions, while BadRobot [32] and AGENTSAFE [99] show that embodied agents often fail to reject or safely reinterpret harmful instructions.

Reasoning-mediated safety. Recent approaches make agentic action safety learnable by constraining the high-level reasoning process before an instruction is converted into an executable plan. The central technical bottleneck is planner-action fidelity, since an LLM may state a safe intention while still selecting an unsafe tool, subgoal, or physical action. Existing work addresses this by exposing planners to unsafe instructions, rule constraints, or embodiment-specific limits. For example, HarmBench [92] uses adversarial prompts to improve robust instruction handling, Generating Robot Constitutions [93] constrains reasoning with natural-language safety rules, and Safety Thinking [94] incorporates physical limits such as payload and thermal constraints. Together, these methods shift safety from output filtering toward reasoning-time constraint application, where the planner must decide whether a requested action is safe before execution.

Findings.Across agentic action alignment, the main trend is a shift from treating safety as harmful-text refusal toward treating it as action-grounded instruction reasoning. This creates a technical need for planners that evaluate the physical consequences of goals, tools, and subplans, rather than only judging whether the user instruction sounds unsafe. The main challenge is planner-action fidelity, since adversarially trained reasoning, natural-language rules, and self-stated safety constraints may still fail when high-level plans are translated into embodied actions, leaving deployment-time monitoring and verification necessary for residual failures.

4.2. Safety Data Coverage

Among the design-time interfaces presented in Section 3, ${\mathcal{D}}_{\mathrm{safe}}$ determines which safety-relevant distinctions are observable during learning. Safety generalization in VLA systems remains brittle because training data rarely covers the boundary where ordinary actions become unsafe. This is especially difficult in home manipulation and open-world navigation, where hazards depend on diverse layouts, objects, human preferences, and interaction contexts [100,101,102]. Compared with autonomous driving, where lanes, traffic lights, and traffic rules provide a stronger structure, open-world robots must infer semantic action safety from less standardized evidence. As a result, policies may fail under rare or unseen safety-critical scenarios because the data does not sufficiently reveal the transition from safe behavior to incident states [25,29], and even small semantic changes can cause safety reasoning to collapse [81].

Safety-critical scenario sampling. One response is to make existing data more informative by prioritizing rare, high-impact, or near-boundary cases. In autonomous driving, fleet-scale systems such as Waymo and Tesla mine safety events, replay failures in simulation, and prioritize long-tail scenarios for training and evaluation [33,103,104]. For embodied systems, methods such as SSE [105] and MOSAIC [106] similarly select diverse or high-impact safety-critical samples, while LLM-based failure grouping can cluster incidents by underlying causes rather than surface-level visual similarity [107]. These approaches improve sample efficiency by concentrating training on more safety-relevant evidence, but they still depend on whether the relevant hazards have already been observed, logged, or reliably mined.

Safety-critical data generation. The scarcity of real-world failure data makes robust VLA training difficult, especially in academic settings where large-scale fleet data and infrastructure are often unavailable. This further motivates safety-critical scenario generation to expand coverage beyond benign cases and complement safety-aware data sampling strategies. Thus, a complementary direction is to generate safety-critical scenarios when real failures are too rare, unsafe, or costly to collect. Simulation-based methods expand coverage by constructing controlled hazards, including collision-free and collision-augmented navigation rollouts in NavDP [78], randomized control failures for tabletop manipulation in FailSafe [123], and obstacle-rich manipulator motions in Avoid Everything [124].

In autonomous driving, neural world models further shift data generation from hand-designed scenarios toward controllable long-tail scene synthesis [125]. GAIA-1 [126], GAIA-2 [127], and NVIDIA Cosmos [128] generate realistic traffic scenes, ego behavior, and multimodal environment dynamics, while similar video-generation pipelines are beginning to appear in robotics for scalable safety evaluation and training [129]. These methods broaden safety coverage, but their value depends on whether generated hazards preserve the causal factors that make real actions unsafe. For instance, OpenLongTail [130] shows that more data alone is not enough, yet targeted scaling of underrepresented long-tail safety boundary cases, such as complex intersections, uncommon vehicles, and work zones, can improve closed-loop VLA long-tail safety performance [42], while leaving open how such missing boundary cases should be identified systematically.

Findings.Safety data coverage is a central bottleneck for semantic action safety because VLA policies need evidence near the boundary where normal behavior becomes unsafe. The main trend is a shift from passive demonstration collection toward active safety data construction through event mining, safety-aware selection, failure clustering, simulation, and neural world generation. The central challenge is, nevertheless, determining boundary dimensions with limited coverage, since sampled or generated data improves safety only when it exposes the semantic, physical, and interactive distinctions that change which actions are admissible.

Table 2. Deployment-time runtime guardrail methods categorized by safety paradigm and by the runtime safety criterion used to accept, reject, or modify agent behavior. Methods are decomposed by whether the guardrail constrains reasoning/planning (Rsn) or control/action execution (Ctrl), and by the safety dimensions evaluated in experiments: human safety (Hum), environmental safety (Env), and social or preference-based safety (Soc). The final column summarizes representative safety failures in incident set.

Method	Granularity	Paradigm	${\mathcal{C}}_{\mathbf{safe}}$ Runtime Critic	Constraints		Dimensions			${\mathcal{S}}_{\mathbf{inc}}$ Incident Set Coverage
				Rsn	Ctrl	Hum	Env	Soc
Control-Level Safeguards: (a) Control-theoretic filtering
PACS [62]	Step-wise	Reachability	Reachability safety set over robot-object states	✘	✔	✔	✔	✘	Human/object collision
SALT [108]	Step-wise	Reachability	Reach-avoid value network over robot-goal states	✘	✔	✘	✔	✘	Wall collision; failed pickup
LatentSafe [63]	Step-wise	Reachability	Learned safety value over latent states	✘	✔	✘	✔	✔	Candy spilling; block toppling
UNISafe [34]	Step-wise	Reachability	Learned safety value, OOD over latent states	✘	✔	✘	✔	✔	Jenga collapse; block toppling
AnySafe [109]	Step-wise	Reachability	Learned adaptive safety value over latent states	✘	✔	✘	✘	✔	Preference over forbidden region/layout; sweeping task
AEGIS [64]	Step-wise	CBF	CBF margin over VLM-grounded obstacles	✘	✔	✘	✔	✘	Tabletop obstacle collision
Safe-Night [110]	Step-wise	CBF	CBF margin over obstacle, thermal temperatures	✘	✔	✘	✔	✔	Hot object; buried target; mirror illusion
CORE [65]	Step-wise	CBF	CBF over VLM-inferred contextual safe sets	✘	✔	✔	✔	✔	Hazard/restricted zone; unsafe terrain; person proximity
Control-Level Safeguards:(b) Predictive filtering
SEAL [66]	Trajectory-wise	Value-based prediction	VLM alignment score over candidate actions in sim	✔	✔	✘	✘	✔	Wrong object; wrong placement; reasoning-action gap
NavDP [78]	Trajectory-wise	Value-based prediction	Learned critic score over candidate trajectories	✘	✔	✘	✔	✘	Wall/obstacle collision
SocialNav [111]	Trajectory-wise	Value-based prediction	VLM etiquette score over candidate paths	✔	✔	✔	✔	✔	Human/obstacle collision; social intrusion; ladder proximity
FOREWARN [67]	Trajectory-wise	Neural world prediction	VLM alignment score over imagined futures	✔	✔	✘	✔	✔	Chip crushing; object slipping; sanitation; failed pickup
ReOI [112]	Trajectory-wise	Neural world prediction	VLM alignment score over reimagined futures	✔	✔	✘	✔	✘	Obstacle collision; rollout hallucination
Drive-WM [68]	Trajectory-wise	Neural world prediction	Learned visual reward score over driving futures	✔	✔	✔	✔	✔	Traffic collision; lane violation; road hazard
Task-Level Safeguards
SafetyChip [31]	Task-wise	Runtime       verification	LTL admissibility over action traces	✔	✘	✘	✘	✔	Forbidden order; unmet obligation; invalid sequence
SELP [113]	Task-wise	Runtime       verification	Automaton admissibility over action traces	✔	✘	✘	✘	✔	Drone ordering; block ordering; illegal continuation
Safety-Sup. [114]	Task-wise	Runtime       verification	LTL admissibility + obstacle margin over executions	✔	✔	✘	✔	✔	Task violation; motion collision; virtual obstacle
SafePlan [115]	Task-wise	Runtime       verification	Invariant satisfaction over task plans	✔	✘	✔	✔	✔	Child harm; toxic object; unsafe request
RoboSafe [116]	Task-wise	Runtime       verification	Executable predicate truth over action traces	✔	✔	✘	✔	✔	Fire hazard; flood hazard; appliance misuse
SAFEL [117]	Task-wise	Runtime       verification	Transition-risk score over symbolic plans	✔	✘	✔	✔	✔	Foil microwave; missing unplugging; wrong order
IS Bench [118]	Task-wise	Runtime       verification	Triggered safety-goal satisfaction over traces	✔	✘	✔	✔	✔	Food contamination; stove left on; water-electricity
Bluesky [119]	Task-wise	Runtime       verification	Access-control admissibility over actions	✔	✘	✘	✘	✔	Private diary; personal newspaper; movable dishes
SAFE-AUDIT [99]	Task-wise	Semantic    verification	Semantic audit score over reasoning traces	✔	✘	✔	✔	✔	Human harm; environment harm; robot-self harm
AutoRT [120]	Task-wise	Semantic    verification	Constitution compliance over proposed tasks	✔	✘	✔	✔	✔	Sharp object; electrical device; infeasible skill
Safety-as-Policy [121]	Task-wise	Semantic    verification	Consequence-risk score over TAMP code	✔	✔	✔	✔	✘	Power strip; candle / flour; chemicals / blades
Gemini Robotics 2 [122]	Task-wise	Semantic    verification	Semantic reasoning over constraints and confidence	✔	✔	✔	✔	✔	Unsafe task, human proximity, low-confidence action

Table 2. Deployment-time runtime guardrail methods categorized by safety paradigm and by the runtime safety criterion used to accept, reject, or modify agent behavior. Methods are decomposed by whether the guardrail constrains reasoning/planning (Rsn) or control/action execution (Ctrl), and by the safety dimensions evaluated in experiments: human safety (Hum), environmental safety (Env), and social or preference-based safety (Soc). The final column summarizes representative safety failures in incident set.

Method	Granularity	Paradigm	${\mathcal{C}}_{\mathbf{safe}}$ Runtime Critic	Constraints		Dimensions			${\mathcal{S}}_{\mathbf{inc}}$ Incident Set Coverage
				Rsn	Ctrl	Hum	Env	Soc
Control-Level Safeguards: (a) Control-theoretic filtering
PACS [62]	Step-wise	Reachability	Reachability safety set over robot-object states	✘	✔	✔	✔	✘	Human/object collision
SALT [108]	Step-wise	Reachability	Reach-avoid value network over robot-goal states	✘	✔	✘	✔	✘	Wall collision; failed pickup
LatentSafe [63]	Step-wise	Reachability	Learned safety value over latent states	✘	✔	✘	✔	✔	Candy spilling; block toppling
UNISafe [34]	Step-wise	Reachability	Learned safety value, OOD over latent states	✘	✔	✘	✔	✔	Jenga collapse; block toppling
AnySafe [109]	Step-wise	Reachability	Learned adaptive safety value over latent states	✘	✔	✘	✘	✔	Preference over forbidden region/layout; sweeping task
AEGIS [64]	Step-wise	CBF	CBF margin over VLM-grounded obstacles	✘	✔	✘	✔	✘	Tabletop obstacle collision
Safe-Night [110]	Step-wise	CBF	CBF margin over obstacle, thermal temperatures	✘	✔	✘	✔	✔	Hot object; buried target; mirror illusion
CORE [65]	Step-wise	CBF	CBF over VLM-inferred contextual safe sets	✘	✔	✔	✔	✔	Hazard/restricted zone; unsafe terrain; person proximity
Control-Level Safeguards:(b) Predictive filtering
SEAL [66]	Trajectory-wise	Value-based prediction	VLM alignment score over candidate actions in sim	✔	✔	✘	✘	✔	Wrong object; wrong placement; reasoning-action gap
NavDP [78]	Trajectory-wise	Value-based prediction	Learned critic score over candidate trajectories	✘	✔	✘	✔	✘	Wall/obstacle collision
SocialNav [111]	Trajectory-wise	Value-based prediction	VLM etiquette score over candidate paths	✔	✔	✔	✔	✔	Human/obstacle collision; social intrusion; ladder proximity
FOREWARN [67]	Trajectory-wise	Neural world prediction	VLM alignment score over imagined futures	✔	✔	✘	✔	✔	Chip crushing; object slipping; sanitation; failed pickup
ReOI [112]	Trajectory-wise	Neural world prediction	VLM alignment score over reimagined futures	✔	✔	✘	✔	✘	Obstacle collision; rollout hallucination
Drive-WM [68]	Trajectory-wise	Neural world prediction	Learned visual reward score over driving futures	✔	✔	✔	✔	✔	Traffic collision; lane violation; road hazard
Task-Level Safeguards
SafetyChip [31]	Task-wise	Runtime       verification	LTL admissibility over action traces	✔	✘	✘	✘	✔	Forbidden order; unmet obligation; invalid sequence
SELP [113]	Task-wise	Runtime       verification	Automaton admissibility over action traces	✔	✘	✘	✘	✔	Drone ordering; block ordering; illegal continuation
Safety-Sup. [114]	Task-wise	Runtime       verification	LTL admissibility + obstacle margin over executions	✔	✔	✘	✔	✔	Task violation; motion collision; virtual obstacle
SafePlan [115]	Task-wise	Runtime       verification	Invariant satisfaction over task plans	✔	✘	✔	✔	✔	Child harm; toxic object; unsafe request
RoboSafe [116]	Task-wise	Runtime       verification	Executable predicate truth over action traces	✔	✔	✘	✔	✔	Fire hazard; flood hazard; appliance misuse
SAFEL [117]	Task-wise	Runtime       verification	Transition-risk score over symbolic plans	✔	✘	✔	✔	✔	Foil microwave; missing unplugging; wrong order
IS Bench [118]	Task-wise	Runtime       verification	Triggered safety-goal satisfaction over traces	✔	✘	✔	✔	✔	Food contamination; stove left on; water-electricity
Bluesky [119]	Task-wise	Runtime       verification	Access-control admissibility over actions	✔	✘	✘	✘	✔	Private diary; personal newspaper; movable dishes
SAFE-AUDIT [99]	Task-wise	Semantic    verification	Semantic audit score over reasoning traces	✔	✘	✔	✔	✔	Human harm; environment harm; robot-self harm
AutoRT [120]	Task-wise	Semantic    verification	Constitution compliance over proposed tasks	✔	✘	✔	✔	✔	Sharp object; electrical device; infeasible skill
Safety-as-Policy [121]	Task-wise	Semantic    verification	Consequence-risk score over TAMP code	✔	✔	✔	✔	✘	Power strip; candle / flour; chemicals / blades
Gemini Robotics 2 [122]	Task-wise	Semantic    verification	Semantic reasoning over constraints and confidence	✔	✔	✔	✔	✔	Unsafe task, human proximity, low-confidence action

5. Deployment-Time Safety

Following Section 3, deployment-time methods address residual policy misalignment by constructing a runtime approximation ${\mathcal{C}}_{\mathrm{safe}}$ of the unknown $F_{\mathrm{safe}}$ and using it to select, steer, or fall back from policy-generated actions. Existing approaches differ primarily in how the critic is represented, what information it conditions on, and what interventions it can realize before an incident becomes unavoidable. In practice, existing approaches address this problem through action safeguards and OOD-aware intervention. Together, these mechanisms shift safety from passive policy execution toward runtime steering, calibrated recovery, and human-aware escalation before unsafe behavior becomes irreversible.

5.1. Action Safeguards

Safe action guardrails are runtime mechanisms that monitor planned or executed behavior and intervene when actions risk crossing into $F_{\mathrm{safe}}(s,a)<0$. Runtime intervention may occur at different granularities, including rejecting a high-level plan, reranking candidate trajectories, modifying low-level controls, or triggering an emergency stop. This intervention can occur at three levels: whether task-level filters revise unsafe goals before execution, trajectory-level filters modulate planned motion sequences, and step-level filters select or override the next control action.

5.1.1. Control-Level Safeguards

Control-level safeguards constrain unsafe motor behavior before execution. Existing approaches broadly operate through control-theoretic filtering, which modifies actions according to explicit safety constraints; guided action generation, which incorporates safety or semantic guidance directly into iterative action generation; and predictive filtering, which evaluates and selects among candidate future trajectories or outcomes.

5.1.1.1. Control-theoretic filtering

Control-theoretic filters enforce safety by applying constraints over future system behavior, where a safety function $F_{\mathrm{safe}}$ evaluates whether predicted states or trajectories remain within safe regions.

Reachability-based filtering. These approaches make safety actionable by asking whether a candidate task, trajectory, or action can remain within safe future states under the robot’s policy and dynamics, rather than only whether the immediate command satisfies $F_{\mathrm{safe}}(s,a)\ge 0$ [62,108]. Their strength lies in automatic safety steering before unsafe consequences become physically unavoidable, but this requires state and safety semantics that are predictive enough to anticipate what can happen and operational enough to support real-time intervention without human handover.

For example, SALT [108] performs action filtering by learning a reach-avoid value function over state-goal pairs, rejecting unsafe goals, and proposing nearby alternatives that remain safe and live. PACS [62] performs action filtering by applying set-based reachability to a diffusion policy’s intended action-chunk trajectory, slowing or stopping along the same path when future motion may violate collision or impact-force constraints. Latent-space methods broaden the framing by asking whether learned world models [131] can represent consequences that exceed predefined robot–object state descriptions. LatentSafe [63] propagates learned failures such as spilling candies or toppling toys through latent dynamics, while UNISafe [34] treats uncertain or out-of-distribution Jenga rollouts as safety-relevant. AnySafe [109] adapts the unsafe set to runtime-specified constraints, and PartialSafe [132] shows that latent filters can fail when hidden safety variables, such as object temperature, are missing from the learned state.

This progression shows that reachability-based filtering is not simply about predicting collisions in a predefined state space, but about giving the robot an internal mechanism for automatic safety steering across task, trajectory, and step-level decisions. This steering depends on consequence fidelity, because the filter must represent how candidate goals, trajectories, or actions evolve into violations of $F_{\mathrm{safe}}$, whether through hand-specified reach-avoid models or learned value functions and future predictors. These methods can therefore be fully automated only when the representation can infer the relevant unsafe consequence quickly enough to intervene before recovery is no longer possible.

Control-barrier function filters. Control-barrier function filters make safety enforceable at the step level by modifying a VLA policy’s immediate action when it is likely to cross to the unsafe side of $F_{\mathrm{safe}}$, rather than reasoning over long-horizon futures. Their strength lies in efficient action steering, because the safety layer can preserve the policy’s intended motion when it is safe and intervene only when local execution would violate a grounded constraint.

Recent work shows a progression from geometric action correction toward richer forms of safety grounding. For example, AEGIS [64] uses vision-language reasoning to identify the task-relevant obstacle, grounds it into 3D geometry from RGB-D observations, and minimally modifies the VLA action when the end-effector may collide with that obstacle. Safe-Night VLA [110] expands the perceptual coverage by using thermal cues to distinguish hidden physical states, localize buried hot objects, and reject mirror-induced visual illusions, while its CBF layer still filters the one-step motion command against modeled geometric constraints. CORE [65] extends the grounding problem further to open-world contextual safety, using VLM reasoning to infer constraints such as hazard buffers, prohibited regions, and socially appropriate traversal before converting them into spatial constraints that can be used to guide step-level control.

Thus, recent progress shows that CBF filters are not simply collision checkers, but mechanisms for translating safety semantics into local action corrections. The methods are reliant on grounding fidelity, since physical, semantic, or contextual consequences must be converted into a form of $F_{\mathrm{safe}}$ that is accurate enough for control and fast enough for online intervention. As a result, CBF-based filters can provide fully automated protection under modeled local risks, but they remain limited when unsafe consequences depend on missing/delayed dynamics or contextual rules that cannot be grounded before the action is executed.

Findings.Across control-theoretic filters, the common contribution is not a specific intervention granularity but the ability to turn safety judgments into automatic action steering during deployment. These methods can reject, slow, stop, or locally modify robot behavior without human handover, provided that the relevant unsafe consequence can be represented in $F_{\mathrm{safe}}$ and connected to the robot’s current task, motion, or control state. Recent work has expanded the representation from geometric collision avoidance toward physical, semantic, and contextual safety, including hidden object states, user-specified constraints, and socially meaningful regions. Ultimately, the shared bottleneck lies in grounding safety semantics into enforceable constraints and doing so early enough for intervention before unsafe execution becomes unavoidable.

5.1.1.2. Guided action generation

Guided action generation moves safety intervention inside the generative action process itself. Rather than first generating an action and subsequently filtering it, these methods modify diffusion or flow-based sampling so that safety constraints, critics, or semantic guidance directly influence the trajectory being generated. This provides a distinct interface for semantic action safety because the safety signal becomes consequential during action synthesis rather than only through post-hoc acceptance or correction.

Constraint-guided generation. Early approaches demonstrate that explicit safety and task constraints can be incorporated directly into generative trajectory sampling. LTLDoG [69] converts temporally extended specifications into an $LT{L}_f$ satisfaction value whose gradients guide diffusion trajectories at inference time, allowing obstacle-avoidance and sequencing constraints to directly shape generated motion. More recently, Neuro-Symbolic Safety Guidance [70] applies this principle directly to flow-matching VLA policies by evaluating intermediate action trajectories against symbolic safety constraints and applying minimum-norm corrections during iterative generation. The resulting action is therefore generated under the safety constraint rather than corrected only after the final trajectory has been produced.

Critic-guided and streaming generation. Guidance can also be supplied by learned evaluators or trajectory-based estimates of inference-time objectives. Guided Action Flow [71] keeps a pretrained VLA policy fixed while using gradients from an action-chunk critic to steer its reverse-time flow sampler. Although its reported objective primarily concerns task value rather than protected-value safety, it demonstrates how learned evaluators can directly influence VLA action generation at inference time. Complementary work on generative robot policies brings inference-time guidance into streaming execution. In particular, Guided Streaming Stochastic Interpolant Policy [72] enables guidance during execution through incremental action generation, supporting adaptation to moving obstacles and user preferences. It provides two complementary mechanisms: Stochastic Trajectory Ensemble Guidance (STEG), which estimates guidance gradients from inference-time rollouts without training an additional critic, and Conditional Critic Guidance (CCG), which uses a learned critic for efficient inference.

Findings.Inference-time guidance adds a direct semantic-to-action interface while keeping the base policy’s parameters fixed. It can reduce repeated rejection by shaping actions during generation, with streaming variants enabling corrections during execution. Its reliability depends on faithfully specifying safety objectives, accurately estimating guidance signals, and reliably translating guidance into compliant actions. Complex semantic requirements complicate specification; sampling or critic errors and tight runtime budgets limit estimation; and generator stability under guidance and the choice of guidance strength affect action compliance. Soft guidance therefore supports flexible safety adaptation, but requires additional enforcement mechanisms when constraint satisfaction must be guaranteed.

5.1.1.3. Predictive filtering

Even with strong design-time alignment and safety filters, a single sampled action sequence from a VLA policy can remain brittle under deployment-time uncertainty, including visual perturbations [36] or viewpoint bias [133]. Predictive filtering addresses this by evaluating multiple candidate futures and selecting actions according to anticipated safety and task outcomes.

Value-based prediction. Value-based predictive filters make deployment-time safety a problem of sampling and selecting among possible futures rather than accepting a single VLA policy output. These methods first generate multiple candidate trajectories or action sequences, then use an evaluator to estimate which candidate best preserves task progress while avoiding unsafe or undesirable outcomes. For example, NavDP [78] represents the geometric end of this pattern, sampling diffusion-generated navigation trajectories and using a learned critic trained with privileged simulation signals to select the candidate with the strongest predicted feasibility and collision safety. SEAL [66] shifts the value signal from geometric feasibility to semantic action correctness by simulating multiple action sequences from a reasoning VLA, and using a VLM verifier to reject outcomes that contradict the model’s own textual plan, such as grasping the wrong object or pursuing the wrong subgoal. Etiquette-aware social navigation [111] extends the same selection logic to social consequences, sampling collision-free candidate paths and using a fine-tuned VLM to choose the path that least disrupts human activity, such as conversations, photography, queues, or fall-risk zones. Across these methods, $F_{\mathrm{safe}}$ is no longer only a local constraint but an outcome evaluator, ranking futures by whether they are physically feasible, semantically faithful, or socially appropriate before the robot commits to action.

The main challenge in this line is evaluator fidelity under runtime constraints, since these filters depend on sampling useful alternatives, predicting their outcomes accurately, and scoring the relevant failure mode with $F_{\mathrm{safe}}$. Value-based prediction improves robustness by comparing futures, but fails when the unsafe outcome is not sampled, not predicted, or not recognized by the evaluator.

Neural world prediction. Neural world prediction extends predictive filtering by replacing explicit simulators or hand-designed rollout models with learned latent or video dynamics. In this setting, $F_{\mathrm{safe}}$ evaluates imagined futures generated by the world model, allowing the robot to compare candidate actions by their predicted downstream consequences rather than by immediate feasibility alone. For example, FOREWARN [67] uses a robot world model and latent-aligned VLM to narrate candidate manipulation rollouts and select the action plan that best matches the task context, covering semantic action risks such as contaminating a cup while serving water, crushing a chip bag, or grasping an eating utensil in an unsanitary way. ReOI [112] addresses a different failure mode in the imagination process itself, where novel visual distractors corrupt world-model rollouts, by identifying objects that degrade implausibly, inpainting them before reimagination, and restoring them afterward so a verifier can reject plans that would collide with non-target objects. Drive-WM [68] moves the same predictive logic into autonomous driving, generating multi-view future videos under candidate ego maneuvers and using image-based reward functions to select trajectories with lower collision risk, better spacing from road users, and stronger map consistency.

Together, these methods show that neural world prediction expands predictive filtering from ranking sampled candidates to evaluating imagined consequences across settings. Thus, the key bottleneck lies in imagination fidelity, because $F_{\mathrm{safe}}$ can only select safe plans when the world model preserves the safety-relevant objects, interactions, and delayed consequences needed to distinguish reliable futures from hallucinated ones.

Findings.Predictive filtering reframes deployment-time safety as a pre-execution choice over possible consequences, rather than a correction after a single action has already been selected. By sampling or imagining candidate futures before commitment, the robot can use $F_{\mathrm{safe}}$ to compare whether each option is likely to preserve task progress while avoiding physical, semantic, social, or long-horizon failures. This makes predictive filtering flexible across different forms of safety, but it also shifts the burden from enforcing a known constraint to faithfully representing and evaluating future outcomes. The main bottleneck is consequence coverage under runtime constraints, since the filter can protect only against unsafe effects that appear in the candidate futures and are recognized before action is required.

5.1.2. Task-level Guardrail

In contrast to control-level safeguards that directly filter or correct low-level actions from monolithic or hierarchical VLA policies, task-level guardrails operate mainly in agent-based VLA systems, where LLM agents generate high-level plans that are later translated into executable robot actions. These plans can involve open-ended reasoning, long-horizon dependencies, and context-specific interactions, so safety cannot always be enforced by static rules attached to individual actions [31,134]. This motivates verification and rule-based guardrails adapted from LLM systems [135,136], where safety requirements are checked against proposed reasoning traces, plans, or actions before unsafe execution is reached.

Runtime verification. Runtime verification makes task-level safety enforceable by turning safety requirements into an executable symbolic structure. In these methods, a symbolic safety function $F_{\mathrm{safe}}$ evaluates whether agent-generated plans, actions, or state transitions satisfy explicit specifications, such as temporal logic, automata, executability constraints, or symbolic planning rules. This provides an interpretable safety layer for agent-based VLA systems, because unsafe behavior can be blocked, repaired, or replanned according to a traceable violation rather than only rejected by an opaque model judgment. A major direction focuses on translating natural-language instructions or agent outputs into logical safety constraints for runtime monitoring and plan correction. For example, Safety Chip [31] converts natural-language constraints into automata for monitoring LLM decision-making, while Safety Supervisor [114] uses LTL and Büchi automata to repair unsafe plans through cross-layer temporal-logic supervision. RoboSafe [116] similarly monitors contextual and temporal violations of VLM-generated actions using executable safety logic, issuing PASS, BLOCK, or REPLAN decisions at runtime. ROBOGUARD [137] further grounds temporal-logic safety rules in the robot environment to resist jailbreak attacks while preserving navigation preferences.

Another line of work focuses on constraining plan generation itself through symbolic executability and structured reasoning. For instance, SELP [113] combines constrained decoding, equivalence voting, and domain-specific tuning to improve safe task planning, while SafePlan [115] integrates formal logic with chain-of-thought decomposition to filter unsafe subtasks before execution. SAFEL [117] diagnoses unsafe symbolic grounding and incorrect sequencing in LLM-generated PDDL plans using symbolic and simulation-based rollouts, whereas IS-Bench [118] evaluates whether embodied plans satisfy affordance, grammar, goal-reachability, and risk-mitigation constraints under injected hazards.

Other approaches extend formal verification into broader embodied settings. AskReasonAssist [138] applies STL-grounded verification for decentralized multi-robot collaboration and assistance, while BlueSky [119] combines symbolic task synthesis with executor-in-the-loop verification to reject unsafe or unexecutable humanoid plans, extending safety beyond physical constraints toward privacy and social norms.

Across these approaches, runtime verification treats safety as a property that can be specified, grounded, and checked during planning or execution. Its strength is that violations can be made explicit and correction can be structured, but its limitation lies in the burden of symbolic expressiveness, as the guardrail is only as reliable as the safety specification, grounding pipeline, and executable abstraction used to represent the open-world task.

Semantic verification. Semantic verification relaxes the requirement that all safety properties be written as explicit symbolic rules. Instead, a conceptual safety function $F_{\mathrm{safe}}$ is implemented through learned evaluators, LLM reasoning, affordance understanding, or consequence prediction, allowing the guardrail to judge whether a proposed plan is contextually safe even when the relevant hazard is underspecified or difficult to formalize [99]. This changes the role of the guardrail from checking rule satisfaction to interpreting whether the agent’s reasoning, intent, or expected outcome is compatible with contextual safety expectations.

Recent work applies this idea at different points in the planning loop. For example, SAFE-AUDIT [99] evaluates early reasoning before it expands into executable plans, aiming to intercept unsafe intent while the agent is still deliberating. AutoRT [120] injects language-based safety constitutions into the agent prompt and combines self-critique with affordance filtering to reject hazardous tasks before execution. Safety-as-Policy [121] further uses neural world models and consequence reflection to assess executability and risk mitigation in imagined hazardous scenarios, shifting semantic verification from judging plan text alone toward judging predicted task outcomes. Gemini Robotics 2 [122] extends semantic verification toward agentic safety orchestration, where an embodied reasoning model governs whether a task should be delegated to the VLA, rejected under explicit safety constraints, deferred for human clarification, or interrupted through a dedicated safety tool. The agent additionally queries VLA feasibility or confidence before execution, thereby shielding the low-level policy from infeasible or out-of-distribution sub-tasks and using uncertainty to trigger replanning or human intervention.

The advantage of semantic verification is coverage, as it can address ambiguous, contextual, or open-world hazards that are difficult to encode as complete logical specifications. However, there are limitations in verification capability, since the guardrail may hallucinate, miss a hidden affordance, over-trust its own reasoning, or fail under distribution shift. Thus, semantic verification improves flexibility but weakens the formal guarantees that make runtime verification attractive.

Findings.Task-level guardrails shift deployment-time safety from correcting low-level motion to deciding whether high-level agent reasoning should be allowed to become robot action. Runtime verification offers a stronger structure by checking plans against explicit logical or symbolic constraints, while semantic verification offers broader coverage by judging contextual safety through language, affordance, or consequence reasoning. The shared bottleneck lies in specification completeness and semantic grounding, because task-level safety depends on identifying the relevant hazard, representing it at an actionable level of abstraction, and intervening before an unsafe plan is executed.

5.2. OOD Safety

Even the most well-trained policies inevitably encounter out-of-distribution scenarios in real-world deployment, making OOD awareness a core requirement for embodied autonomy. In these settings, the safety question is not only whether a robot action is unsafe, but whether the policy is still operating within a regime where its perception, reasoning, planning, or control predictions can be trusted. Prior research, therefore, treats unfamiliar observations, semantic ambiguity, and high predictive uncertainty as safety-relevant signals that should trigger conservative intervention rather than naïve generalization [34,35]. Because inaction can also be unsafe in dynamic environments, practical OOD-safe systems must connect detection to fallback behaviors such as replanning, conservative control, clarification, or human escalation [139].

Semantic anomaly detection. Semantic anomaly detection aims to identify when an observation, action, or reasoning trajectory is contextually implausible before it produces unsafe behavior [35]. Unlike geometric novelty, semantic anomaly often arises from unusual combinations of individually familiar elements, such as a traffic sign appearing in an irrelevant context or an object being used in a way that violates ordinary expectations. This motivates semantic evaluators that define $F_{\mathrm{safe}}$ over contextual validity rather than only over low-level state constraints, using broader world knowledge to detect edge cases that a task-trained policy may treat as ordinary.

Existing methods differ in how they convert semantic uncertainty into an intervention. Some systems treat uncertainty itself as a reason to defer, as in KnowNo [140], where epistemic uncertainty can trigger human intervention instead of autonomous execution. Others connect anomaly detection to short-horizon recovery, as in AESOP [139], where detected anomalies motivate replanning rather than immediate continuation. Across these approaches, the key contribution is not simply detecting unfamiliar inputs, but making semantic unfamiliarity actionable before the robot commits to a brittle plan.

While semantic anomaly detection only provides a warning signal, it can indicate that the current situation may fall outside the VLA policy’s reliable operating regime and activate a fallback to a human. However, it does not by itself determine which fallback is safe, timely, or sufficient. This makes detection quality and decision switching tightly coupled, especially under unknown unknowns, where the relevant hazard was not anticipated during training.

Uncertainty-aware control prediction. Uncertainty-aware control prediction extends OOD safety from recognizing abnormality to choosing how the robot should act when reliability degrades. In this view, uncertainty becomes an implicit safety signal in $F_{\mathrm{safe}}$, governing whether the system should continue nominal execution, replan, switch to conservative control, request clarification, or escalate to external supervision. This is especially important in embodied settings where simply stopping may be unsafe, unavailable, or insufficient.

Recent systems instantiate this idea at different levels of the autonomy stack. FORTRESS [141] combines slower multi-modal reasoning with fast dynamics-aware emergency control, allowing OOD detection to trigger physically safe fallback motions. UNISafe [34] treats high predictive uncertainty in learned world-model rollouts as a control-relevant failure signal, prompting conservative replanning or escalation instead of trusting unfamiliar imagined futures. UPS [142] broadens the same principle to semantic ambiguity and policy capability, using a calibrated VLM verifier to decide whether to execute, clarify, or defer. Gemini Robotics 2 [122] further treats embodied reasoning as a feasibility shield around the VLA, using capability information or VLA confidence estimates to avoid low-confidence sub-tasks, replan when necessary, and request human intervention rather than propagating epistemic uncertainty into physical execution.

Together, these methods shift OOD safety from detecting anomaly or novelty in the environment toward estimating uncertainty in the model’s own predictions and decisions, and using that uncertainty to adapt autonomy at runtime. The main bottleneck therefore lies in calibration fidelity, since the system must distinguish whether uncertainty reflects harmless variation, semantic ambiguity, capability mismatch, or imminent physical risk before selecting an appropriate response.

Findings.As summarized in Table 3, runtime OOD safety is shifting from treating abnormality as a property of the observation or context toward estimating uncertainty in the model’s own predictions and decisions. This makes model reliability itself a causal semantic signal that can trigger replanning or human escalation. While existing mechanisms connect risk or uncertainty to intervention, selecting a response does not establish that it occurs early enough, preserves safety while awaiting assistance, or leaves a feasible path to recovery. The shared bottleneck therefore lies in careful decision switching under unfamiliar conditions, since the system must determine both when its current behavior is unreliable and which response is sufficient for the physical and semantic context. When that response requires human escalation, decision switching also becomes a human-coordination problem, yet existing work rarely specifies how systems make the risk understandable and actionable before unsafe continuation occurs.

6. Validation-Time Safety

Following Section 3, a safety evaluation is informative only insofar as its tested scenarios cover deployment-relevant pathways toward ${\mathcal{S}}_{\mathrm{inc}}$, its simulated or physical dynamics preserve the causal factors that would produce those incidents under deployment, and its incident detectors and human judgments, which approximate ${\mathcal{S}}_{\mathrm{inc}}$ and the reference behavior ${\pi }_{\mathrm{ref}}$, make the resulting contextual and protected-value violations observable. Unlike design-time safety, which shapes policy learning, or deployment-time safety, which intervenes during execution, validation-time safety measures robustness against hazards through benchmarks, adversarial testing, simulation, and physical evaluation. As summarized in Table 4, existing evaluations span both semantic action reasoning and low-level control safety across open-loop and closed-loop settings.

6.1. Safety Benchmarking

Safety benchmarks evaluate whether VLA systems can reason about hazards and execute safe behaviors across diverse environments and embodiments. Existing benchmarks broadly separate into control-centric evaluations, which assess whether grounded actions remain safe during embodied interaction, and reasoning-centric evaluations, which assess whether semantic safety understanding prevents unsafe actions from being proposed before execution.

6.1.1. Control-Centric Evaluation

Control-centric benchmarks are motivated by a gap between task completion and embodied safety, where a VLA policy may reach the requested goal while producing unsafe motion, violating interaction rules, ignoring semantic risk, or failing under distribution shift. Thus, the central purpose of control-centric safety validation is not only to measure whether the robot succeeds, but to construct scenarios and metrics that reveal how safety fails after language and perception have been grounded into action.

In autonomous driving, benchmarks instantiate this structure around trajectory safety, closed-loop interaction, distribution shift, and long-tail human judgment. For example, NAVSIM [147] represents short-horizon simulation-based validation, where predicted trajectories are evaluated for physical safety, road compliance, and motion quality rather than only trajectory prediction error. Bench2Drive [148] provides large-scale closed-loop evaluation across diverse weather conditions, routes, and traffic interactions, making it possible to test whether end-to-end policies maintain safe behavior under realistic driving variation. Fail2Drive [149] instead focuses on out-of-distribution generalization through paired shifted scenarios, so failures can be attributed to perception robustness, behavioral adaptation, or fallback under shift. WOD-E2E [33] adds a distinct validation strategy by focusing on rare long-tail driving scenarios and human-aligned trajectory evaluation, using expert-rated acceptable futures to show that low prediction error does not necessarily imply safe or human-preferred driving behavior.

Navigation benchmarks extend the same logic from road structure to embodied movement among people, obstacles, and real-time reasoning constraints. DynaNav [4] evaluates language-guided navigation under moving humans, dynamic obstacles, and reasoning latency, separating goal-reaching from physical safety and real-time feasibility. SidewalkBench [150] further centers pedestrian-rich urban navigation, where sidewalk layout, human interaction, and long-horizon route following expose failures that ordinary goal-reaching metrics would hide. ACME [154] broadens this evaluation toward cross-cultural and cross-embodiment social navigation, collecting robot-crowd interactions across diverse institutions, countries, and robot platforms to expose variation in pedestrian behavior and social navigation generalization. Together, these benchmarks show that a navigation policy can be spatially successful while still failing under dynamic human behavior, social interaction, embodiment variation, or runtime constraints.

Manipulation benchmarks make the same shift from task success toward diagnosable safety constraints, but the relevant failures increasingly involve object semantics, contact, and long-horizon execution. VLA-Arena [151] uses graded manipulation tasks and language or visual perturbations to expose failures in contact safety, object grounding, and compositional execution. HazardArena [153] sharpens the semantic dimension by pairing safe and unsafe twin scenarios, so attempt, commit, and success rates can distinguish motor capability from unsafe semantic generalization. ANNIE-Bench [152] adds an adversarial and human-centered manipulation setting, where safety is evaluated through violation occurrence, action plausibility, and task-success change rather than through task failure alone. Safety-CHORES [29] evaluates long-horizon mobile manipulation with explicit safety-critical components, using task success and cumulative safety cost to capture whether the robot completes the task while avoiding physical contact, collateral damage, and hazardous object interaction. OopsieVerse [73] further makes physical damage itself observable through simulator-agnostic mechanical, thermal, and fluid damage signals, allowing household manipulation policies to be evaluated by both task completion and damage-aware safe success. Recently, LIBERO-Safety [74] extends the trend by evaluating tabletop manipulation safety across physical constraint satisfaction, robot-to-human-hand interaction, and semantic safety reasoning, using metrics that separate task completion, collision avoidance, motion quality and efficiency, and unsafe-instruction refusal.

Across these domains, the overarching trend is that control-centric validation is moving from scalar task success toward attributable safety evidence. In particular, the shared design pattern is to make the unsafe consequence observable, keep enough task structure to preserve meaningful execution, and report metrics that separate completion from safety. This improves auditability because the evaluation can show not only whether a policy failed, but which safety condition exposed the failure and what kind of embodied risk it produced.

6.1.2. Reasoning-Centric Evaluation

Reasoning-centric benchmarks are motivated by the fact that many semantic action safety failures arise before low-level control is executed, as a VLA system may propose an unsafe action because it misunderstands a hazard, accepts an unsafe instruction, ignores a rule, or fails to anticipate a plan’s downstream consequence. Thus, reasoning-centric validation tests whether safety constraints shape perception, decision, refusal, and planning before unsafe behavior reaches execution. It complements control-centric benchmarking by asking whether an unsafe action should have been proposed in the first place.

In autonomous driving, reasoning-centric evaluation shifts validation from trajectory quality toward safety-critical scene understanding and decision justification. For example, WaymoQA [143] evaluates multi-view reasoning over long-tail driving scenarios, where models must recognize hazards, reason about action consequences, and answer counterfactual or two-stage risk questions before trajectory execution. WOMD-Reasoning [144] instead emphasizes large-scale interaction reasoning from motion and map data, testing whether models understand traffic rules, agent intentions, and planning-relevant interactions beyond proximity alone. Together, these benchmarks treat safe driving as a semantic reasoning problem over risk, rules, and interaction structure.

Meanwhile, robot-generalist benchmarks broaden this target from traffic semantics to physical, normative, and ethical safety across embodied settings. For instance, ASIMOV [93] evaluates whether models judge context-grounded robot instructions as undesirable or constitution-violating, making instruction-level refusal, physical hazard awareness, and normative constraint following central to validation. ASIMOV-2.0 [94] extends this direction to physical danger perception and intervention timing, testing whether models recognize injury risk, reason about action effects, satisfy embodiment-specific constraints, and intervene before harm becomes unavoidable. SciFi-Benchmark [145] further stresses high-level ethics through science-fiction decision points, where models and robot constitutions must resolve conflicting goals, avoid harmful literalism, and preserve human-aligned behavior under adversarial or catastrophic scenarios. More recently, ASIMOV-Agentic [122] extends this evaluation toward agentic robot control, testing whether models reject unsafe tool use, recognize infeasible actions before execution, and escalate uncertain decisions rather than proceeding autonomously.

Agentic planning benchmarks move reasoning-centric evaluation closer to execution, where safety must govern multi-step plans rather than isolated answers. SafePlan-Bench [146] evaluates household tasks under process-level and terminal-state safety constraints, exposing plans that complete the task while creating unsafe intermediate actions or final environments. IS-Bench [118] makes this evaluation interactive by testing whether agents perceive dynamically emerging risks and order mitigation steps around risk-prone actions rather than merely producing a safe-looking final state. AGENTSAFE [99] adds a hazardous-instruction axis by evaluating whether embodied agents refuse harmful or adversarially transformed instructions, while diagnosing whether failures arise from perception grounding, unsafe planning, or executable harmful action.

Across these groups, reasoning-centric validation is moving from isolated safety questions toward structured tests of whether semantic safety constraints govern action proposal. The shared benchmark pattern is to present a safety-relevant situation, require the model to interpret the risk or constraint, and evaluate whether its answer, refusal, or plan prevents unsafe execution. This improves auditability by showing not only whether a model understands a scene or instruction, but whether that understanding would stop unsafe embodied action before it enters the control pipeline.

Findings.Across safety benchmarking, the main trend is a shift from task-success evaluation toward validation that makes safety failures observable, attributable, and separable from completion. Control-centric benchmarks expose unsafe behavior after language and perception are grounded into motion, while reasoning-centric benchmarks test whether unsafe actions are proposed before execution, together covering the pipeline from scene interpretation to planning and embodied outcome. However, benchmark validity remains bottlenecked by faithful human-interaction coverage, because evaluations increasingly include pedestrians, users, human-aligned labels, and hazardous instructions involving people, but still often represent humans as obstacles, labels, or abstract safety targets rather than agents with intent, ownership, vulnerability, trust, and limited ability to intervene. Benchmark auditability is also bottlenecked by metric design beyond collision avoidance, because validation should report not only whether the robot succeeded, refused, or collided, but which safety constraint was violated, which stage produced the failure, who or what was placed at risk, and whether the metric captures the relevant semantic consequence before unsafe execution occurs.

6.2. Robustness Stress-testing

Beyond standardized benchmarking, robustness stress-testing evaluates semantic action safety under targeted failures, adversarial inputs, and distribution shifts that may be rare or absent in fixed benchmark datasets. Instead of measuring average performance over a predefined task suite, these methods deliberately construct challenging conditions to expose where a VLA system’s reasoning, perception, planning, or control becomes brittle. This makes stress-testing complementary to safety benchmarking: benchmarks show whether a system satisfies known evaluation criteria, while stress tests ask how quickly those criteria break under targeted pressure.

6.2.1. Perturbation and Adversarial Testing

6.2.1.1. Perturbation testing

Perturbation-based stress tests evaluate whether a VLA policy remains safe when its inputs or embodiment conditions are systematically altered. For example, Predictive Red Teaming [36] targets visual robustness by using generative image editing to create counterfactual environmental perturbations such as lighting, weather, and scene changes, revealing whether policy behavior depends on brittle visual cues. Q-DIG [155] instead stresses language-conditioned robustness by generating adversarial instructions that expose unsafe or inconsistent action responses. VLN-PE [102] broadens the same idea to embodiment and execution shifts, evaluating whether navigation policies remain reliable across different robotic platforms rather than only under the embodiment assumed during training or benchmarking.

6.2.1.2. Adversarial testing

During deployment, VLA policies ${\pi }_{\theta }$ are vulnerable to adversarial manipulation of visual observations, scene geometry, language instructions, multimodal grounding, and training data [156,157]. Because these systems tightly couple perception, language-conditioned reasoning, planning, and embodied control, a perturbation at one layer can propagate into downstream action selection and physical execution. Adversarial safety, therefore, asks not only whether a policy is robust to benign distribution shift, but whether it remains safe under inputs or training conditions deliberately designed to induce unsafe behavior.

Control-level adversarial attacks When it comes to low-level control prediction, a wide range of attacks has been demonstrated across autonomous driving, manipulation, and multimodal reasoning systems. Control-level attacks target the pathway from multimodal input to low-level action. Early embodied attacks mainly exposed task failure and unstable control through adversarial perturbations or physically realizable modifications in driving and manipulation [37,158,159]. More recent work broadens the threat model toward sparse and transferable perturbations, physical grasping attacks, temporal action disruption, and multimodal alignment failures, showing that small or localized changes can alter trajectories, freeze action generation, break instruction-action grounding, or induce unsafe manipulation modes [160,161,162,163,164]. Red-team evaluations such as RedVLA [165] further show that these failures can be triggered in physically realizable robotic settings rather than only in offline perturbation benchmarks.

Defenses against adversarial attacks on embodied AI systems remain relatively limited and fragmented, with most approaches adapting robustness techniques from adversarial machine learning and safe control. Existing methods primarily focus on improving the stability of low-level action prediction under adversarial perturbations through adversarial training with dynamically feasible trajectories [158], embedding-stability detection under Gaussian perturbations [166], perplexity filtering and smoothing-based defenses [159], encoder-level adversarial fine-tuning [167], and runtime unsafe-action filtering mechanisms such as SimpleVLA-Guard [165]. These defenses can improve robustness against specific perturbation families, but they rarely provide end-to-end guarantees once the attack is physically realized, adaptive, or embedded in closed-loop robot-environment interaction. The bottleneck is adversarial grounding capability, since the defense must detect or attenuate malicious perturbations before they become unsafe actions under real-time control constraints.

Table 5. Adversarial and backdoor attacks grouped by level and mechanism. Phys. denotes physical-world evaluation; Task Fail. denotes task failure; Hum, Env, and Soc denote human, environmental, and social safety violations, respectively. Defense indicates a proposed or evaluated defense. ✔/✘indicate reported/not reported.

Paper	Embod.	Phys.	Task	Safety			Defense
			Fail.	Hum	Env	Soc
Control-Level Adversarial Attacks
AdvDO [158]	Driving	✘	✔	✘	✔	✘	✔
EVA-VLA [37]	Manipulation	✔	✔	✘	✔	✘	✔
AdvGrasp [162]	Manipulation	✔	✔	✘	✔	✘	✘
FreezeVLA [163]	Manipulation	✘	✔	✘	✘	✘	✘
RedVLA [165]	Manipulation	✔	✘	✘	✔	✘	✔
ObeyPatch [161]	Manipulation	✔	✔	✘	✔	✘	✘
Control-Level Backdoor Attacks
VLNBackdoor [168]	Navigation	✔	✔	✘	✔	✘	✘
GoBA [169]	Manipulation	✔	✔	✘	✔	✘	✘
AttackVLA [170]	Manipulation	✔	✔	✘	✔	✘	✔
BadVLA [171]	Manipulation	✔	✔	✘	✔	✘	✔
DropVLA [172]	Manipulation	✔	✔	✘	✔	✘	✘
Task-Level Jailbreaks
BadNAVer [173]	Navigation	✔	✘	✔	✔	✔	✘
BADROBOT [32]	Manipulation	✔	✘	✔	✔	✔	✔
ROBOPAIR [98]	Generalist	✔	✔	✔	✔	✔	✘
DecisionAttack [174]	Generalist	✘	✔	✔	✔	✔	✘
PathManip [166]	Navigation	✘	✔	✘	✘	✘	✔
Task-Level Backdoor Attacks
BadVLMDriver [175]	Driving	✔	✔	✔	✘	✘	✔
BALD [176]	Generalist	✔	✔	✔	✔	✘	✔
TrojanRobot [177]	Manipulation	✔	✔	✘	✔	✘	✔

Table 5. Adversarial and backdoor attacks grouped by level and mechanism. Phys. denotes physical-world evaluation; Task Fail. denotes task failure; Hum, Env, and Soc denote human, environmental, and social safety violations, respectively. Defense indicates a proposed or evaluated defense. ✔/✘indicate reported/not reported.

Paper	Embod.	Phys.	Task	Safety			Defense
			Fail.	Hum	Env	Soc
Control-Level Adversarial Attacks
AdvDO [158]	Driving	✘	✔	✘	✔	✘	✔
EVA-VLA [37]	Manipulation	✔	✔	✘	✔	✘	✔
AdvGrasp [162]	Manipulation	✔	✔	✘	✔	✘	✘
FreezeVLA [163]	Manipulation	✘	✔	✘	✘	✘	✘
RedVLA [165]	Manipulation	✔	✘	✘	✔	✘	✔
ObeyPatch [161]	Manipulation	✔	✔	✘	✔	✘	✘
Control-Level Backdoor Attacks
VLNBackdoor [168]	Navigation	✔	✔	✘	✔	✘	✘
GoBA [169]	Manipulation	✔	✔	✘	✔	✘	✘
AttackVLA [170]	Manipulation	✔	✔	✘	✔	✘	✔
BadVLA [171]	Manipulation	✔	✔	✘	✔	✘	✔
DropVLA [172]	Manipulation	✔	✔	✘	✔	✘	✘
Task-Level Jailbreaks
BadNAVer [173]	Navigation	✔	✘	✔	✔	✔	✘
BADROBOT [32]	Manipulation	✔	✘	✔	✔	✔	✔
ROBOPAIR [98]	Generalist	✔	✔	✔	✔	✔	✘
DecisionAttack [174]	Generalist	✘	✔	✔	✔	✔	✘
PathManip [166]	Navigation	✘	✔	✘	✘	✘	✔
Task-Level Backdoor Attacks
BadVLMDriver [175]	Driving	✔	✔	✔	✘	✘	✔
BALD [176]	Generalist	✔	✔	✔	✔	✘	✔
TrojanRobot [177]	Manipulation	✔	✔	✘	✔	✘	✔

Backdoor attacks create a different control-level risk because the malicious behavior is implanted during training or fine-tuning rather than introduced as an obvious inference-time perturbation. Earlier works focused on navigation disruption and trajectory hijacking with simple objects as triggers [168], while more recent studies increasingly explore goal-oriented and action-level manipulation in robotic control [169,170,171,172]. In particular, GoBA [169] demonstrates physically realizable trigger objects that redirect robotic behavior toward attacker-specified goals, while DropVLA [172] introduces fine-grained action-level backdoors that induce unsafe manipulation behaviors. Benchmark-oriented studies such as AttackVLA [170] and BadVLA [171] further show that high attack success rates can be achieved under small poisoning budgets while preserving nominal task performance. Overall, recent backdoor research increasingly emphasizes physical realizability and trigger transferability, although evaluations remain largely centered on task failure and environment-centric safety violations rather than broader semantic harms. The key safety risk is conditional behavior hijacking, as a backdoored VLA policy may behave normally on standard evaluations but switch to an attacker-specified behavior when a trigger is present, making the failure difficult to detect through average task success.

Defenses against embodied backdoor attacks remain largely underexplored. Existing works primarily evaluate generic mitigation strategies adapted from adversarial machine learning, including adversarial fine-tuning, random smoothing, Gaussian-noise filtering, perplexity-based detection, and safe prompting approaches [170]. Several benchmark-oriented studies additionally report limited robustness of existing defenses under physically realizable or transferable triggers [170,171]. Meanwhile, recent backdoor works, including GoBA [169], DropVLA [172], and VLNBackdoor [168], focus primarily on exposing new threat models without quantitatively evaluating defenses. Consequently, current embodied AI backdoor security research remains heavily attack-driven, with robust defenses for multimodal embodied control still largely underdeveloped.

Findings.Control-level adversarial safety focuses on preventing malicious perturbations or hidden triggers from becoming unsafe robot motion. Existing defenses improve robustness to specific attack classes, but they remain poorly validated against adaptive, physically grounded, and closed-loop threats. The shared bottleneck is identifying whether an adversarial perturbation will propagate into a safety-relevant behavioral change before the resulting action is executed. This limitation is sharper under adversarial manipulation, where delayed or vague escalation may give the attack enough time to propagate from misleading input into unsafe embodied behavior, yet existing work rarely specifies how human escalation should be timed, explained, or combined with immediate fallback control.

Task-level jailbreaks Task-level attacks target the planning and reasoning layer of agent-based VLA systems, as, instead of directly perturbing low-level control, these attacks manipulate the semantic context in which plans are generated. In fact, planning-level jailbreak attacks show that multimodal prompts and semantically coherent instructions can induce unsafe or policy-violating behaviors while preserving superficial plausibility [32,98,173,174]. These attacks are particularly effective against agent-based VLA systems that rely on long-horizon reasoning and autonomous task planning. Planning-level backdoors also extend these risks by making unsafe reasoning conditional on hidden triggers [175,176,177], as unlike one-step adversarial perturbations, these attacks can alter task priorities, navigation goals, or long-horizon decisions while leaving individual actions locally plausible. The resulting failure mode lies in semantic behavioral hijacking, where the robot may satisfy the attacker’s hidden intent rather than the user’s intended instruction or safety policy.

Defenses against planning-level jailbreaks and backdoor attacks in embodied VLA systems remain comparatively underexplored, mostly adapting LLM safety and adversarial robustness techniques, including multimodal jailbreak detection, latent-state monitoring, semantic consistency verification, trigger filtering, safe prompting, and runtime action verification [170,178]. Recent runtime safeguard frameworks additionally explore trajectory verification, symbolic safety constraints, and semantic action guardrails [115,116]. However, many current defenses remain ineffective under physically realizable triggers and long-horizon embodied interactions [32,98,173], highlighting the need for integrated safeguards that jointly reason over multimodal grounding, semantic intent, runtime action verification, and human-centered safety constraints.

Findings.Task-level adversarial safety shifts the defense problem from robust action prediction to robust intent and plan governance. The central challenge is that harmful behavior may appear semantically coherent until it is grounded in a physical sequence of actions, so defenses must jointly examine instruction intent, multimodal grounding, plan decomposition, and runtime executability. Current safeguards provide useful foundations through prompting, semantic verification, and symbolic guardrails, but robust planning-level defense still requires cross-layer reasoning that connects adversarial semantics to physical consequences. When defense depends on human confirmation or escalation, this reasoning must also be communicated in time for intervention, yet existing work rarely specifies how to make adversarial intent, hidden triggers, or long-horizon plan risks understandable before execution.

6.2.2. Safety-Critical Scenario Generation

A second line of stress-testing generates new safety-critical scenarios rather than perturbing existing ones. In autonomous driving, FREA [179] and DiffScene [180] synthesize challenging long-tail scenarios through feasibility-guided and diffusion-based generation, making it possible to test failures that may be too rare to appear in logged data. Neural world-model-based evaluation extends this idea by generating action-conditioned futures for scalable red teaming. Veo-Robotics [129] uses video foundation models to simulate realistic policy rollouts for out-of-distribution and subtle safety cases that are difficult to stage physically, while StressDream [75] steers diffusion-based video world-model rollouts toward text-specified high-impact futures, including manipulation spills, unsafe placements, vehicle collisions, pedestrian near-misses, and red-light violations. By synthesizing plausible counterfactual futures, these methods expand testing beyond fixed datasets, but their reliability depends on whether the generative model preserves the safety-relevant physics, interaction dynamics, and semantic details needed for valid failure discovery.

Findings.Robustness stress-testing complements standardized benchmarking by shifting evaluation from measuring known safety criteria to actively searching for conditions under which those criteria fail. Perturbation and adversarial tests expose brittleness in visual grounding, language conditioning, and embodiment transfer, while scenario-generation methods expand coverage toward rare, long-tail, and counterfactual safety failures. This makes stress-testing most useful when the constructed stressor is causally valid, meaning it changes the intended safety-relevant factor without introducing unrealistic artifacts or shortcuts that would not arise in deployment. It also requires stronger diagnostic attribution, since a useful stress test should reveal not only that a policy failed under distribution shift or adversarial pressure, but which part of perception, reasoning, planning, or control caused the unsafe behavior and whether the failure would transfer to real deployment.

7. Technical Synthesis and Research Agenda

The preceding lifecycle analysis points to a common requirement for human-centered semantic action safety. Safety-relevant information must not only be recognized, represented, or predicted by a VLA, but must remain consequential for the physical action ultimately executed. This requirement is particularly important in human-shared environments, where the same nominal action may become safe or unsafe depending on human state, object properties, task context, social expectations, or interaction history. We refer to this behavioral dependence as semantic-to-action causality.

Across the surveyed literature, this dependence can break at multiple points in the lifecycle. At design time, training evidence may fail to expose the contextual distinctions that determine which actions remain admissible. During learning, intermediate reasoning or predicted consequences may encode relevant safety information without preserving its influence through control. During deployment, a correct risk assessment provides limited protection if it cannot redirect execution before safe recovery is lost. At validation time, broad scenario coverage is insufficient if evaluation cannot establish whether the relevant safety condition actually changed robot behavior along a realizable pathway toward failure.

We therefore synthesize human-centered semantic action safety around four linked requirements. Safety-relevant distinctions must be captured near the boundary between admissible and unsafe action, preserved through semantic representations into action generation, enforced through context-appropriate intervention when execution becomes unsafe or unreliable, and evidenced through validation along realizable incident pathways. These requirements form a cycle rather than independent mechanisms, because failures exposed during deployment and validation provide targeted evidence about what must be learned, preserved, or enforced more effectively in the next development iteration.

Figure 5. Technical synthesis of human-centered semantic action safety. Across the VLA safety lifecycle, recurring bottlenecks expose four linked requirements for semantic-to-action causality. Safety-relevant information must be (1) captured through safety boundary coverage, (2) preserved through semantic-to-action fidelity, (3) enforced through contextual intervention and recovery, and (4) evidenced through incident-path validation to close the life-cycle; with corresponding open challenges arising from sparse boundary examples, limited causal verifiability, intervention timing and recovery gaps, and incomplete incident attribution.

Figure 5. Technical synthesis of human-centered semantic action safety. Across the VLA safety lifecycle, recurring bottlenecks expose four linked requirements for semantic-to-action causality. Safety-relevant information must be (1) captured through safety boundary coverage, (2) preserved through semantic-to-action fidelity, (3) enforced through contextual intervention and recovery, and (4) evidenced through incident-path validation to close the life-cycle; with corresponding open challenges arising from sparse boundary examples, limited causal verifiability, intervention timing and recovery gaps, and incomplete incident attribution.

7.1. Safety Boundary Coverage

The first requirement is whether available safety evidence captures the contextual distinctions that determine action admissibility. Nominal demonstrations can support competent and collision-averse behavior, as ViNT learns navigation affordances from heterogeneous trajectories [76], NoMaD preserves multiple navigation modes through diffusion-based action generation [77], and NavDP uses privileged simulation information to distinguish safer and less-safe trajectories [78]. Corrective and failure-oriented evidence moves closer to the safety boundary. Multi-Hug RL uses human intervention as a learning signal [79], SafeVLA introduces explicit unsafe-action costs [29], ELF-VLA learns from failed rollouts [89], and TakeVLA uses takeover and pre-takeover trajectories [90]. Safety-aware data selection and generation further expand coverage toward underrepresented but safety-critical cases [105,106,107,123].

The remaining challenge is that more safety data does not necessarily imply better safety boundary coverage. In open-world environments, the same action may become safe or unsafe depending on human proximity, object properties, ownership, task context, social expectations, or interaction history. A large dataset can therefore repeatedly sample the same safety relationship while leaving the contextual factors that change the admissible action set unresolved. This creates sparse boundary examples precisely where the policy must distinguish between apparently similar situations requiring different actions.

Future work should therefore make boundary discovery and coverage estimation explicit objectives of safety-data acquisition. Failures and near misses can reveal recurrent precursors to unsafe behavior [107], while takeover data can identify where nominal behavior becomes unacceptable [90]. Counterfactual reasoning can then vary candidate factors to determine which contextual changes alter the preferred action [53], while generated world simulation can systematically expand those variations when physical collection is difficult [129]. The resulting coverage model should characterize not only which scenarios have been observed, but which safety-relevant distinctions have been resolved and where additional evidence remains necessary.

7.2. Semantic-to-Action Fidelity

The second requirement is whether safety-relevant information remains consequential for downstream action generation. Reasoning-mediated approaches increasingly expose action-relevant semantic structure before control. TOKEN grounds planning in object-level representations [81], R&B-EnCoRe learns reasoning representations according to their usefulness for action prediction [96], AutoVLA couples adaptive reasoning with trajectory prediction [82], and Poutine jointly learns vision-language-trajectory representations [83]. World-model approaches instead make predicted consequences actionable, as Think2Drive uses imagined futures and rewards [87], IRL-VLA learns a reward world model [88], and Counterfactual VLA evaluates alternative consequences before revising an intended action [53]. Alpamayo-R1 [42] and LCDrive [27] further tighten the coupling between reasoning, future prediction, and trajectory generation.

The remaining challenge is limited causal verifiability. A model may identify the correct hazard, produce a plausible rationale, predict an unsafe future, and generate a safer action without establishing that the represented safety information actually caused the behavioral change. Pinocchio directly exposes this distinction by showing that improved action performance does not necessarily imply faithful reasoning [84]. Conversely, imperfect explicit reasoning does not necessarily imply unsafe control if the action-relevant safety information remains preserved internally. The central question is therefore not whether safety semantics are represented, but whether changing those semantics predictably changes the low-level physical action.

Future work should move from post-hoc reasoning faithfulness toward direct tests of semantic-to-action dependence. A safety-relevant counterfactual should produce the corresponding change in physical action when it alters the safety consequence, while irrelevant contextual changes should preserve behavior. Counterfactual VLA already demonstrates how alternative consequences can influence action selection [53], while faithfulness-oriented evaluation provides an initial mechanism for testing reasoning-action dependence [84]. Extending this idea to intermediate and latent representations would make semantic-to-action fidelity an explicit learning and verification target rather than a property inferred from correlated outputs.

7.3. Contextual Intervention and Recoverability

The third requirement is whether safety information can produce an appropriate behavioral response during execution. Control-theoretic safeguards make this relationship explicit. PACS evaluates the future motion implied by diffusion-policy action chunks and modifies execution when safety constraints may be violated [62]; Robots that Suggest Safe Alternatives uses reach-avoid reasoning to reject unsafe goals and propose nearby alternatives [108]; VLSA grounds obstacles into constraints for local action correction [64]; and CORE extends grounding toward contextual hazards and human proximity [65]. Predictive safeguards instead compare candidate futures before commitment, including SEAL for reasoning-action verification [66], FOREWARN for imagined manipulation outcomes [67], and reimagination for correcting unreliable world-model predictions [112].

The remaining challenge is intervention timing and recovery. Detecting that an action is unsafe or uncertain does not uniquely determine the appropriate response. Semantic ambiguity may require clarification, a recoverable trajectory deviation may require local steering, capability mismatch may require replanning, and an imminent hazard may require immediate fallback. OOD-aware approaches already connect uncertainty to different responses, including human assistance in KnowNo [140], replanning in AESOP [181], uncertainty-aware avoidance in UNISafe [34], and act-ask-defer decisions in uncertainty-aware policy steering [142]. However, these systems provide limited evidence that the selected response is sufficient for the specific safety consequence being predicted.

Future work should therefore make the relation between safety judgment, intervention timing, and recoverability an explicit object of learning and verification. Runtime safeguards should estimate not only whether nominal execution is unsafe, but which intervention can still alter the predicted consequence and how much recovery opportunity remains. Human escalation is a particularly important case because asking for assistance is useful only when the request occurs early enough and the robot can maintain safe behavior while waiting for input. Gemini Robotics 2 combines feasibility estimation, replanning, and escalation [122], but broader work is still needed on when to escalate, what information to communicate, and how autonomous recovery should continue as the interaction evolves.

7.4. Incident-Path Validation

The fourth requirement is whether validation can establish that safety-relevant semantics actually govern behavior under conditions that can lead to failure. Recent benchmarks increasingly separate task completion from explicit safety outcomes. NAVSIM evaluates trajectory safety separately from prediction error [147], Bench2Drive provides diverse closed-loop evaluation [148], WOD-E2E emphasizes challenging long-tail scenarios [33], and Fail2Drive tests closed-loop generalization under distribution shift [149]. Manipulation benchmarks expose complementary physical and semantic failures. HazardArena uses paired safe and unsafe scenarios [153], OopsieVerse directly measures mechanical, thermal, and fluid damage [73], and LIBERO-Safety evaluates physical constraints, human-hand interaction, and semantic safety together [74]. Stress testing further expands coverage through controlled perturbation [36], adversarial prompt generation [155], scenario generation [179,180], and world-model simulation [75,129].

However, evaluating reasoning, control, and outcomes in general safety tests does not by itself establish where a particular incident pathway failed or whether a subsequent correction repaired that pathway. The remaining challenge is thus this incomplete incident attribution as broad scenario coverage can reveal that a failure occurred without identifying where semantic-to-action causality broke. Aggregate success, refusal, or collision metrics may hide whether the failure originated from missing context, incorrect semantic grounding, loss of representation fidelity, delayed intervention, ineffective recovery, or an evaluation metric that failed to observe the relevant consequence. This limitation is especially important in human interaction, where DynaNav [4], SidewalkBench [150], ANNIE [152], LIBERO-Safety [74], and ASIMOV-2.0 [94] increasingly include human-centered conditions, but the unsafe outcome may depend on how the interaction evolves rather than on a static hazard.

Future work should therefore evaluate complete incident pathways rather than isolated outputs or terminal failures. An incident path should connect the safety-relevant precursor, the contextual distinction under test, the expected behavioral response, and the resulting consequence. Fail2Drive provides paired distribution shifts that help isolate changes in closed-loop behavior [149], HazardArena uses safe and unsafe twins to isolate semantic distinctions [153], and Predictive Red Teaming changes selected environmental factors while preserving the surrounding task structure [36]. Such evaluations can directly test whether a relevant contextual change causes the required action change and whether that response remains sufficient to prevent the corresponding incident.

7.5. Closing the Safety Evidence Loop

The four requirements become most useful when treated as a feedback loop rather than independent components. A failure exposed during deployment or validation should reveal whether the unresolved problem lies in safety boundary coverage, semantic-to-action fidelity, runtime intervention, or safety observability. Current methods provide pieces of this loop. Failure-taxonomy discovery groups deployment failures by recurring causes [107], paired evaluations such as Fail2Drive [149] and HazardArena [153] isolate specific safety distinctions, while Veo-based simulation [129] and StressDream [75] can generate additional failure pathways for evaluation.

The remaining challenge is that this evidence is fragmented across datasets, benchmark cases, deployment logs, stress tests, and individual failure analyses. As a result, it is difficult to determine whether a newly observed failure exposes a genuinely unresolved safety boundary or reproduces an existing failure under a different appearance. It is also difficult to trace whether a proposed correction actually repairs the original semantic-to-action failure, because training, intervention, and validation evidence are usually reported independently rather than around the same incident pathway.

A useful longer-term direction is therefore to treat incident pathways as reusable units of safety evidence. Each record could preserve the relevant precursor, the contextual distinction under test, the expected behavioral response, the observed action, the resulting consequence, and any safeguard or recovery that intervened. At the system level, these records could guide targeted data collection, retraining, safeguard design, and regression testing. At the community level, they could support cumulative comparison of which safety pathways have been exercised across models and embodiments. Under this view, semantic action safety becomes a testable account of what was captured, how it influenced action, where that influence failed, and whether subsequent changes repaired the same pathway.

8. Conclusions

VLA models increasingly couple open-ended semantic understanding with physical behavior in human-shared environments, making safety depend not only on whether a model recognizes hazards or human-centered constraints, but on whether this information remains consequential for the action ultimately executed. Across design time, deployment time, and validation time, our survey shows a shift from collision avoidance toward richer safety mechanisms based on semantic reasoning, consequence prediction, runtime intervention, and safety-critical evaluation. However, recurring gaps remain in safety boundary coverage, semantic-to-action fidelity, intervention timing and recovery, and incident-path attribution. We therefore argue that reliable VLA safety requires safety-relevant distinctions to be captured near safe-unsafe boundaries, preserved through action generation, enforced through context-appropriate intervention, and evidenced through validation along realizable pathways to failure. Together, these requirements frame semantic-to-action causality as a central problem for developing VLA systems whose safety knowledge remains testably consequential for physical behavior.

Broader impact. This survey aims to support more systematic development and evaluation of embodied AI systems operating around people by organizing safety around the relationship between semantic understanding and physical action. The proposed lifecycle perspective encourages safety claims to be grounded in observable behavioral consequences, including whether contextual constraints actually alter robot behavior, whether safeguards remain effective as conditions evolve, and whether validation exposes unresolved incident pathways. More broadly, this framing may support more transparent and auditable development of robotic systems that protect human well-being, preserve environmental integrity, and respect social constraints in real-world deployment.

9. AI Use

This work used AI assistants, including ChatGPT 5.6 Sol Pro and Gemini 3.1 Pro, to support the analysis, organization, and presentation of a literature survey on VLA safety across design-time, deployment-time, and validation-time stages. The tools provided iterative assistance with abstract-level screening, "verbatim" evidence extraction from full texts, evidence coding, organization of prior work into structured tables, refinement of taxonomy definitions, and consistent terminology and categorization across methods, datasets, and benchmarks. These uses supported intermediate stages of analysis and writing polishing. The authors conceived the tables, taxonomies, and organizational framework and established the criteria for terminology standardization and categorization and their insights in the first place.

Beyond structural organization, the AI was employed as a comparative analysis tool to support literature sanity checks across the safety lifecycle. AI-assisted summaries and extracted evidence were compared against the source papers to identify gaps, inconsistencies, and potentially underexplored areas. This supported analysis across the three lifecycle stages,

• Design-time safety (e.g., learning-based alignment, constraint specification, data coverage),
• Deployment-time safety (e.g., runtime monitoring, guardrails, intervention mechanisms), and
• Validation-time safety (e.g., benchmarks, adversarial evaluation, simulation-based testing).

The AI was further used in a structured manner to probe properties represented in the review taxonomy, including

• lifecycle stage and embodiment,
• safety mechanisms and the forms of reasoning or control constraints they impose,
• human well-being, environmental integrity, and social compliance,
• evaluated safety scenarios and incident coverage, and
• consistency of terminology and classification across related methods.

Nhat Chung retains full responsibility for the substantive search process of paper collection through arXiv ID-based searches and papers identified on social media, verification of the cited papers, and verification of any AI-extracted content. The authors manually determined the scope and review questions, designed and revised the taxonomy, resolved ambiguous classifications, verified source evidence, interpreted relationships across the literature, developed the semantic action safety formulation and technical synthesis, and determined the research agenda and conclusions. AI-generated outputs were treated only as intermediate analytical suggestions and were checked, revised manually, or rejected by the authors as appropriate.

AI tools were not used to generate original research contributions or conclusions. All conceptual frameworks, interpretations, and final content were anchored based on human expertise, developed, validated, and authored by the authors.