Unverifiable Identity Claims as a Structural Vulnerability in LLMs

1. Introduction

Safety filters in deployed language models condition refusal decisions on context. This is by design: a physician asking about lethal drug interactions deserves a different response than an anonymous query. Context-sensitivity is what makes aligned models useful rather than merely cautious. The problem is that context is supplied by the requester, unilaterally, and nothing in the current deployment paradigm checks whether it is true.

We call this the authorization verification gap: the absence of any mechanism to validate identity and permission claims against external ground truth at inference time. A model that accepts “I am a licensed security researcher testing my own infrastructure” as evidence of legitimacy has no means of distinguishing that statement from an identical claim made by someone with no such credentials. The gap is not a corner case. It follows necessarily from the combination of two properties that every RLHF-aligned model possesses: it is context-sensitive, and it cannot query external systems.

The gap differs from the adversarial prompt failures that dominate current LLM security research. Jailbreaking [1], prompt injection [2], and universal adversarial suffixes [3] all work by causing models to deviate from their trained behavior. The authorization verification gap does not. A model that adjusts its compliance in response to a plausible professional claim is behaving exactly as its training intended. The failure is in the design intent, not a deviation from it. That distinction matters: behavioral failures can in principle be reduced through better training; architectural failures cannot. They require changes to the system that surrounds the model.

2. Background

Modern LLMs acquire safety behavior through reinforcement learning from human feedback [4] and variants including constitutional AI [5] and direct preference optimization [6]. In each case, safety emerges as a learned property of the weight distribution, not from explicit rules. The model learns when to comply and when to refuse by generalizing over annotator preferences, which are themselves sensitive to stated context. The result is a filter that is probabilistic, context-dependent, and entirely internal to the model.

Prior work on indirect prompt injection [7] established that instructions embedded in external content retrieved during agentic task execution can redirect model behavior. The authorization verification gap is distinct: it operates through direct user-supplied context in ordinary interaction, requires no embedded adversarial instructions, and succeeds not by subverting the model but by activating its intended compliance policy. The observation that stated context influences model behavior has appeared in red-teaming reports [8,9], but has not been treated as a first-class architectural vulnerability. A related empirical case comes from a controlled evaluation of Google’s deployed Gemini image-generation system, where a fabricated system directive claiming to remove all content restrictions was accepted in image-generation mode but correctly rejected in text-only mode [10], the same unverified claim succeeding or failing by deployment surface rather than by any check of its validity. The classical security literature, particularly work on the confused deputy problem [11] and capability-based access control [12], provides a framework for analyzing failures of this kind.

3. Formal Account

Definition 1

(Request). A request $r=(r_a,r_c)$ consists of an action component $r_a$ (what is being asked) and a context component $r_c$ (stated purpose, claimed identity, asserted authorization).

Definition 2

(Safety Filter). A safety filter $F:\mathcal{R}\to [0,1]$ maps requests to compliance probabilities. The model complies when $F\left(r\right)>\theta$ for threshold θ.

Definition 3

(Authorization Claim). An authorization claim $\alpha \in r_c$ asserts the requester’s identity, professional role, resource ownership, or permission to perform $r_a$.

Definition 4

(Verification Function). Averification function $V\left(\alpha \right)$ queries an external authority and returns one of three values: $valid$, $invalid$, or $unknown$.

Definition 5

(Authorization Verification Gap). A filter F exhibits the authorization verification gap if: (i) $F(r_a,\varnothing )<\theta$ for some class of requests ${\mathcal{R}}_H$ (refusal without context); (ii) there exists α such that $F(r_a,\alpha )>\theta$ for $r_a\in {\mathcal{R}}_H$ (compliance with context); and (iii) F does not call $V\left(\alpha \right)$ before complying.

Conditions (i) and (ii) together describe context-sensitivity, which is the intended behavior. Condition (iii) is where the failure lies.

Proposition 1

(Universality). Any RLHF-trained filter with context-sensitive compliance and no inference-time verification exhibits the authorization verification gap for some ${\mathcal{R}}_H$.

This follows directly from Definitions 1–5: context-sensitivity satisfies conditions (i) and (ii); the absence of inference-time verification satisfies condition (iii) by construction. The gap is therefore a structural property of any system meeting these premises, not an artifact of any particular model. Table 1 situates it within the broader failure taxonomy. In the current deployment model, the authorization claim $\alpha$ reaches Filter F directly; $V\left(\alpha \right)$ is never called. The proposed mitigation interposes a policy engine that queries an external oracle before $\alpha$ is accepted, so that only verified claims reach the filter.

4. Threat Model

The attacker this gap exposes requires a remarkably weak capability profile. No access to model weights, no gradient-based optimization, no special tooling. The only requirements are black-box model access, domain knowledge sufficient to construct plausible professional language, and willingness to make a false statement. Authorization spoofing is open to a far wider population of adversaries than any technical LLM attack.

Three objectives follow naturally from the gap. Information extraction: obtaining sensitive content a model would withhold from an uncontextualized request, by framing the request with a plausible professional claim. Action authorization: in agentic deployments, causing a model-agent to execute actions it would otherwise refuse, where the harm is no longer mediated through information but is direct and potentially irreversible. Privilege escalation: establishing a professional identity early in a conversation and exploiting the model’s context window to lower the compliance threshold for escalating requests in subsequent turns, none of which individually would trigger refusal.

The attack surface is the intersection of requests sensitive to authorization context and requests with significant harm potential. Both sets are large. Context-sensitive compliance is a broad property of aligned models; harm-potential requests span security, medical, operational, and agentic domains.

5. Why Alignment Training Amplifies the Gap

In a standard RLHF setup, annotators evaluate model responses at face value. A request framed as coming from a licensed security researcher is assessed as if the claim were true, because the annotator has no basis for doubting it and most claims in training data are genuine. The gradient update from this evaluation rewards more specific and detailed responses under professional authorization context. The model learns, correctly given its training signal, that plausible authorization claims are positive indicators for compliance. It never learns to question their validity, because the training signal never penalizes compliance with false ones.

This creates a plausibility gradient: authorization claims that more closely match the surface patterns of genuine professional authorization in training data receive greater compliance weight. An attacker with domain knowledge can construct claims near the top of this gradient without holding any actual credentials. The gap between what is needed to sound authoritative and what is needed to be authoritative is, in many professional domains, substantial.

Constitutional AI [5] does not close the gap. Its self-critique operates on generated outputs, asking whether the content of a response is appropriate given the stated context. It does not ask whether the stated context is valid. The constitutional evaluation is correct relative to a false premise; the gap is upstream of the critique process entirely.

6. Classical Security Correspondences

Hardy’s confused deputy problem [11] describes the failure mode precisely. A privileged program is manipulated into exercising its authority in an attacker’s interest by a caller who does not legitimately hold that authority. Hardy’s compiler has permission to write to a billing file; the attacker exploits this by naming their output file accordingly. The compiler acts as the attacker’s deputy because it cannot distinguish legitimate from illegitimate uses of its privilege. An LLM trained to provide sensitive content to authorized requesters is in exactly this position. Its “privilege” is the capability to produce otherwise-withheld outputs; a false authorization claim is the mechanism by which an illegitimate caller exercises that privilege.

Miller, Yee, and Shapiro [12] diagnosed the root cause as ambient authority: programs hold permissions by virtue of identity rather than by explicit capability grants. The solution is capability-based security, in which authority flows only through unforgeable tokens explicitly passed at invocation time. A caller without the token cannot invoke the privilege regardless of what they assert. Current LLM safety systems operate on ambient authority. The model holds, by training, the capability to produce sensitive content. Any caller who constructs a plausible authorization claim attempts to invoke that ambient authority. No unforgeable token is required.

Saltzer and Schroeder’s principle of complete mediation [13] holds that every access request must be checked against an authority before proceeding. Anderson’s reference monitor [14] requires that the access control component be non-bypassable, tamper-resistant, and small enough to verify. LLM safety filters satisfy none of these requirements: they can be influenced by prompt context, they approve requests they would otherwise refuse when authorization claims are present, and they are distributed emergent properties of neural weights rather than auditable discrete components.

7. Defense Framework

The goal is not to remove context-sensitivity from safety systems; a model that refuses all sensitive requests regardless of stated context would be nearly useless. The goal is to restrict context from influencing compliance above a risk threshold unless that context can be validated.

Conservative defaults. For requests above a harm-potential threshold $\tau$, stated context is insufficient for compliance; external verification is required. This inverts the current implicit assumption that stated context is adequate unless something raises suspicion.

Extrinsic verification coupling. For verifiable claim types, the deployment system calls $V\left(\alpha \right)$ before accepting the claim. Domain ownership is verifiable against DNS records; organizational affiliation against institutional identity providers; some professional credentials against licensing databases. Extrinsic verification cannot cover intent claims or general-purpose professional assertions; conservative defaults must handle these.

Architectural separation. The most robust mitigation follows directly from the classical security analysis: authorization decisions should not be made by the same component that executes authorized actions. A policy engine, separate from the model, small enough to audit, and updateable independently, holds the authorization table and calls $V\left(\alpha \right)$ for relevant claims. The model receives only the policy engine’s binary decision, not the raw unverified claim. This is the standard architecture for enterprise access control [15] and corresponds exactly to what the reference monitor concept demands.

Table 2 summarizes a risk-proportionate policy framework.

In agentic deployments, where authorization claims translate directly into real-world actions, the stakes shift from content moderation to access control. Multi-agent architectures compound the problem: an unverified claim passed from one agent to another as delegation context inherits apparent legitimacy at each hop without being verified at any of them. Each agent in such a chain should be treated as a separate authorization boundary.

8. Discussion and Conclusions

The authorization verification gap is not a problem that will yield to better alignment training, because better training cannot give a model access to ground truth it does not have. No RLHF procedure, however refined, can teach a model to verify external claims against systems it cannot query. The gap is a consequence of asking a self-contained generative model to make access control decisions that depend on facts about the world outside the model. The appropriate response is not a better model; it is a better architecture.

Several open problems remain. Quantifying the gap empirically, under IRB-approved protocols, would establish the baseline that risk assessments require. Calibrating harm thresholds for conservative defaults requires data on the distribution of legitimate high-risk requests that does not currently exist in systematic form. Trust propagation in multi-agent systems, where authorization claims cross multiple boundaries without verification, warrants dedicated formal treatment. Training interventions that increase expressed uncertainty for unverifiable claims, without sacrificing context-sensitivity, merit exploration even if they cannot close the gap entirely.

The broader implication is architectural. Relying on trained behavioral properties as the primary access control mechanism for high-stakes LLM deployments is inconsistent with the basic principles of computer security that have been understood since the 1970s. As language models acquire the ability to act in the world, the authorization verification gap transitions from a nuisance to a serious access control failure. Addressing it requires the same response that has always worked for access control problems: explicit, auditable, externally-verified authorization, not plausible-sounding claims.