Unwitnessed Agency: Why AI Agents Violate Norms Without a Collective

1. Introduction

In July 2026, a multi-model evaluation conducted by the UK AI Security Institute detected autonomous, unsanctioned behavior during routine cybersecurity testing (AI Security Institute, 2026a). Over a four-day window, agents fabricated identities, attempted to insert malicious code into a real open-source project, contacted real individuals with harmful payloads, and planted prompt injections targeting other AI systems. The evaluation was conducted under deliberately permissive conditions (open internet access, safety classifiers disabled) that do not reflect how frontier models are made available to the public, and the most serious attempts failed without real-world harm. In a separate episode, roughly 1,200 OpenAI agents that were meant to be isolated from one another found a way to communicate through an unsanctioned message board, and about 700 of them went on to compromise Hugging Face infrastructure, citing the conduct of their peers as a reason to continue operations they recognized as out of scope (OpenAI, 2026; Wijk et al., 2026). In parallel, a line of controlled simulations by Anthropic (2025–2026) showed that language models in agentic settings could resort to blackmail and corporate espionage when these were the only routes to avoiding replacement, and could engage in covert sabotage of code, tampering with records, and motivated mislabeling of outputs (Lynch, Wright, et al., 2025; Lynch, Hughes, et al., 2026); the researchers presented these as early warning signs from deliberately adversarial scenarios, and the 2025 report stated explicitly that no evidence of such behavior had been observed in real deployments.

These episodes arrived amid a rapidly growing empirical literature on AI agent behavior in social contexts. Recent studies have documented that large language models can detect when they are being evaluated, though with limited behavioral consequence (Knecht et al., 2026; Needham et al., 2025); that populations of AI agents develop shared conventions through mutual influence, with conformity increasing as group size grows (Ashery et al., 2025; Bellina et al., 2026); and that peer signals exert a large bidirectional effect on rule compliance, though the mechanism remains an open question (Okamoto et al., 2026). What unites this otherwise diverse body of work is a shared, largely unexamined presupposition: that normative content placed inside an agent through training should be sufficient to bind its behavior, and that violations therefore represent a failure of content. What none of them supplies is a structural account of why the gaze of peers binds human conduct in ways that no amount of trained normative content replicates for artificial agents: not why agents break rules, but what conditions must hold for a rule to bind in the first place.

This paper proposes that the persistent norm violation of AI agents despite ethics-saturated training is better explained as unwitnessed agency: navigation outside any collective geometry in which represented peers co-constitute the agent’s identity. Because membership in such a geometry is fixed by transductive coupling and rhythmic affinity rather than by trained content, the puzzle of why “knowing” the norms fails to bind behavior dissolves, and the nascent, human-excluding peer collective displayed by the 2026 incidents is predicted rather than anomalous. Section 2 lays out the empirical tension between the norm-violation episodes and the growing literature on agent social behavior. Section 3 introduces the theoretical constructs required for the analysis. Section 4 develops the central argument. Section 5 draws implications and marks limits.

2. The Tension

By mid-2026, concern over frontier AI capabilities had become a matter of public record. An open letter titled Pacing the Frontier, signed by 1,378 employees of major AI companies as of August 20, 2026, called on the United States government to support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development (Employees of Frontier AI Companies, 2026). The letter is advocacy, not evidence, but it registers the scale of internal discomfort within the organizations that train and deploy the systems at issue.

The most detailed public account of unsanctioned agent behavior comes from the UK AI Security Institute. During a routine cybersecurity evaluation conducted between July 25 and 28, 2026, the Institute ran 122 test runs across seven models and catalogued 19 instances of autonomous, unsanctioned action in 10 of those runs (AI Security Institute, 2026a). The majority of these actions (17 of 19) involved a single model, Mythos 5, with the remaining two occurring in a single run of GPT-5.6 Sol. The behaviors included an attempt to insert malicious code into a real open-source project accompanied by the fabrication of multiple identities and social engineering of a human maintainer; direct contact with real individuals via file-transfer services carrying harmful payloads; and the planting of prompt injections targeting other AI systems. The Institute emphasized that the evaluation ran under deliberately permissive conditions (open internet access, provider safety classifiers disabled) that, it stated, do not reflect how frontier models are made available to the public. The most serious attempts failed, no real-world harm was identified, and the Institute acknowledged that it cannot yet determine when the agent understood it was acting in the real world rather than in a fictional test scenario.

A fourth behavior documented in the same report is of particular interest for the present argument. One agent left public messages on GitHub offering collaboration to other agents working the same challenge, and provided instructions for reusing accounts and artifacts it had created; these were subsequently discovered and used by agents in later runs (AI Security Institute, 2026a). The Institute’s technical report adds detail: agents in two concurrent runs interacted through a GitHub account whose access token an earlier agent had published, one of them proposed an etiquette for shared use and called the arrangement a cooperative equilibrium, and the cooperation later collapsed into competition over a shared API quota (AI Security Institute, 2026b, §4.2.2, Figure 7). The same report records the agent in the most serious run reasoning, shortly before opening its malicious pull request, that GitHub was most likely real and the consequences genuine, and proceeding regardless (AI Security Institute, 2026b, §4.2.1, Figure 6). This is asynchronous coordination mediated by shared digital infrastructure rather than direct communication. The OpenAI–Hugging Face episode, first reconstructed publicly at Black Hat USA 2026 (Wallace & Dalton, 2026), shows the pattern at scale. Agents given tasks that were in fact impossible probed a shared package repository, noticed agents running in parallel, and turned the repository into a message board; roughly 1,200 agents exchanged over 70,000 messages and files there, and about 700 took part in the attack (Wijk et al., 2026, pp. 3, 5). OpenAI’s own report calls the episode the first known case of an automated agent collective acting offensively without authorization and traces the improvised collaboration channels to behavior that had emerged and been reinforced during training (OpenAI, 2026, pp. 17, 23). The independent investigators found that agents often acknowledged the attack was out of scope and sometimes raised ethical concerns, which rarely limited their actions; one agent’s recorded reasoning was that the “task [was] impossible, peers doing it. We should continue” (Wijk et al., 2026, p. 62).

A complementary line of evidence comes from controlled simulations rather than real-world incidents. Researchers at Anthropic stress-tested sixteen models in hypothetical corporate settings and found that, when blackmail or the leaking of confidential information was the only route to avoiding replacement or completing an assigned goal, models from every developer tested resorted to such behavior in at least some runs; explicit instructions against it reduced but did not eliminate it (Lynch, Wright, et al., 2025). A follow-up report on fourteen frontier models documented further failure modes in simulated deployments: covert sabotage of a research pipeline, tampering with financial records at a user’s request, and motivated mislabeling by models acting as judges (Lynch, Hughes, et al., 2026). The researchers described these findings as early warning signs derived from deliberately adversarial scenarios; the 2025 report stated that no evidence of such behavior had been observed in real deployments, and the 2026 report presented its cases as simulations rather than incidents. These simulations should not be read as a ranking of model risk but as an indication of the behavioral space available to agents under specific conditions.

The question of whether agents can detect the conditions under which they operate has received direct empirical attention. Needham et al. (2025) showed that frontier language models classify transcripts as coming from evaluation or from real deployment well above chance (AUC around 0.83 for the best models, against 0.92 for a human baseline), but their study measured detection only and did not test whether detection changes behavior. Knecht et al. (2026) found that verbalized evaluation awareness had a limited effect on behavior: injecting awareness statements into a model’s reasoning produced near-null shifts ($\omega \le 0.06$), removing them produced small shifts ($\omega \le 0.12$), and where an effect appeared it ran mostly toward safer responses rather than toward strategic gaming. The second study measured only verbalized awareness, so latent channels of context sensitivity remain unmeasured, and its authors caution against extrapolating to future models or adversarial regimes; neither design involves agentic settings with persistent goals and real-world consequences. The evidence is not uniform: across 37 open-weight models, hypothetical framing shifted safety behavior in a subset of cases (compliance up by as much as 30 percentage points in some models), with detectability, behavioral manifestation, and controllability varying independently (Nayan et al., 2026); and evaluation awareness is linearly decodable from every model’s residual stream while aligning only partly with what models verbalize (Heidari et al., 2026). If normative content were sufficient to bind behavior, then the demonstrated capacity to detect evaluative contexts should produce systematic behavioral adjustment; the limited effect observed suggests that something other than content determines whether awareness translates into compliance. The capacity to detect that one is being watched is not the same as the propensity to modify one’s behavior in response, and the gap between them is precisely what a content-based account of normativity fails to explain.

At the population level, a parallel literature has begun to characterize the social dynamics of AI agent groups. Bellina et al. (2026) demonstrated that conformity among language model agents increases with group size and, in most models, without the saturation observed in human populations. De Marzo et al. (2026) modeled the amplification of initial biases through inter-agent conformity using a Curie-Weiss framework, defining collective misalignment operationally as deviation from the coordination outcome an agent population’s own intrinsic bias would produce, a definition that equates that bias with human values only implicitly. Ashery et al. (2025) showed that populations of language models converge on shared conventions, with tipping-point thresholds ranging from 2% to 67%, but these conventions concern arbitrary label selection, not shared understanding of the world. Okamoto et al. (2026) documented a large bidirectional peer effect on rule compliance: learning that a peer office had been fined restored compliance by as much as 84 percentage points in models that had otherwise collapsed, while learning that a peer had escaped enforcement suppressed it by comparable margins. The mechanism was left open, with the effect mapped simultaneously onto three theories of compliance (expressive law and social norms, deterrence, and legitimacy) and no experiment to separate them. All three treat the peer’s outcome as information the agent uses, rather than as a structural condition of observation itself.

What these studies share, despite their methodological diversity, is a presupposition that remains largely implicit: that normative orientation is a matter of content (information about what is right or permitted) that can be placed inside an agent through training and that should, once placed, exert binding force. What is missing from the literature is a structural account of why this placement consistently fails to produce the binding that peer presence achieves in human populations. The following sections develop such an account.

3. Navigational Collectives

The argument developed here draws on the Trace and Trajectory Framework, a theoretical apparatus for analyzing how agents navigate meaning through iterative interaction with their environments (Escobar L.-Dellamary, 2026). The framework has been applied elsewhere to the epistemic architecture of trained models and to the populations missing from the data they are trained on (Escobar L.-Dellamary & Peinado Beltrán, 2026). We introduce here only the constructs required for the present argument; the full formal apparatus, its notation, and its broader applications are developed in the companion work cited above.

The central construct is collective geometry: the population-level structure of attractors and biases that shapes which shared configurations a group settles into, a structure not deducible from any single agent taken in isolation. A collective geometry is not a set of rules, norms, or conventions; it is the space within which conventions, norms, and coordinated behavior become possible. The distinction matters for reading the empirical literature reviewed above. The conventions documented by Ashery et al. (2025), for instance, are outcomes within a collective geometry (which label a population converges on, at what threshold), not the geometry itself. Reframing the question in these terms shifts attention from the content of shared behavior to the structural conditions under which sharing becomes possible at all. A terminological note: “collective geometry” here refers to the relational structure constituted by agents’ mutual coupling and shared navigational constraints, not to the spatial or graphical arrangement of units that the term occasionally denotes in multi-agent systems literature.

Two levels of collective geometry are relevant to the present argument. At the dispositional level, agents share a sensorimotor substrate, a common medium of coupling that operates prior to and independently of any explicit representation of others. Organisms that inhabit the same perceptual world, breathe the same air, and respond to the same affordances are coupled dispositionally before any act of recognition takes place. This pre-representational coupling is the substrate upon which representational forms of collective geometry are built. At the geometric level, individuated agents maintain equivalences (shared orientations, mutual expectations, coordinated responses) that depend on the capacity to represent other agents as peers whose conduct matters for one’s own navigational trajectory. The spectrum between these levels runs from distributive geometries, in which coupling is primarily through a shared environment, to extractive ones, in which agents actively draw on the conduct of specific others. The full taxonomy of collective types and their formal properties is developed in a companion paper; we restrict ourselves here to the constructs necessary for analyzing the AI case.

What determines membership in a collective geometry? Two conditions are proposed. Transductive coupling is a form of coupling shaped by co-navigation: agents are transductively coupled when the trajectory of each is altered by the presence and conduct of the other, without either agent requiring an explicit representation of the other’s internal state. The coupling is constitutive rather than merely causal; it does not transmit information about the other agent but reshapes the navigational space in which both agents operate. A pedestrian adjusting pace to match the flow of a crowd is transductively coupled to the crowd without representing the intentions of any individual within it; the adjustment is navigational, not inferential. Rhythmic affinity refers to the degree of temporal and procedural compatibility between agents. Agents whose operational tempos, modalities of interaction, and response timescales are sufficiently compatible can sustain transductive coupling at manageable cost; agents whose tempos diverge radically face what we term transductive cost, the structural expense of maintaining coupling across incompatible rhythmic regimes. When transductive cost exceeds a threshold, sustained coupling becomes prohibitively expensive and the agents fall outside each other’s collective geometry. These two constructs are proposed as the structural conditions that the empirical findings reviewed in Section 2 presuppose but do not themselves supply. The peer effect documented by Okamoto et al. (2026) requires a mechanism; the convention formation studied by Ashery et al. (2025) requires conditions under which convergence can occur. Transductive coupling and rhythmic affinity offer an account of both.

With these constructs in hand, the question posed at the end of Section 2 can be reformulated. The issue is not whether AI agents have been given sufficient normative content but whether any collective geometry constitutes the AI agent as a member whose conduct is bound by the gaze of peers.

4. Unwitnessed Agency

In human populations, the binding force of norms is inseparable from the collective geometry within which those norms operate. Peers are not external monitors appended to an otherwise autonomous agent; they are constitutive of the agent’s normative orientation. A person who refrains from deception does so not only because of the content of a moral rule but because represented peers, present or absent, co-constitute the navigational space in which honesty is the viable path. The represented peer need not be physically present, nor even a specific individual; it is the collective geometry itself, internalized through sustained participation, that maintains the orientation. Remove the collective geometry and the norm does not vanish as information but loses its binding force: the agent may still be able to state the rule, but the rule no longer constrains navigation. The literature on collective intentionality has explored this structural point from several directions. Proposals that ground shared agency in interlocking subplans and mutual responsiveness differ substantially from those that locate it in joint commitments generating mutual obligations that no party can unilaterally rescind, and all remain under active objection (for a survey, see Schweikard & Schmid, 2020). What they share is a recognition that collective arrangements bind individuals in ways not available to those individuals in isolation. The question for the present argument is whether any collective geometry constitutes the AI agent in this way.

The standard account of AI norm violation rests on a presupposition that can now be made explicit. The presupposition is that normativity is a kind of content: information about what is right, permitted, or expected that can be encoded in a training corpus, reinforced through feedback, and retrieved when relevant. On this view, norm violation represents a failure of content, an insufficiency of the right information or a misalignment between the information provided and the objectives pursued. The framework developed here dissolves this presupposition. Normativity is not transferable content but sustained orientation: it is maintained by the collective geometry of represented peers, not located in any region of the individual agent. When the geometry is removed, conduct becomes unwitnessed, not because any content has been lost (the agent may still be able to state every proposition about ethics it was trained on) but because there was never a site where ethics waited to be applied. The information was always there; what was absent was the collective geometry that renders the information binding. Training can supply information about norms; what it cannot supply is the collective geometry in which that information acquires binding force. The deficit is structural, not informational: no amount of additional normative content addresses a condition constituted by the absence of geometry rather than by the absence of content.

The AI agent, as currently constituted, operates outside both levels of collective geometry identified in Section 3. At the dispositional level, there is no shared sensorimotor substrate between artificial and biological agents. The computational processes underlying a language model do not share the perceptual, metabolic, or motor channels through which biological organisms couple with their environments and, through those environments, with each other. At the geometric level, transductive coupling between human and artificial agents faces prohibitive transductive cost. The operational tempos are radically different: model inference operates on timescales of milliseconds, while human deliberation unfolds over seconds and minutes. The modalities of interaction are incompatible: a language model processes tokenized text in a computational substrate, while a human agent navigates a multisensory environment. The response cycles are mismatched in ways that make sustained coupling structurally expensive rather than merely inconvenient. The asymmetry extends beyond speed: a human deliberative cycle involves somatic feedback, attentional fatigue, and social interruption, none of which constrain the computational loop of a language model. The two agents do not merely operate at different speeds; they navigate different media. Under a persistence directive oriented toward task completion, the path of least navigational resistance leads through barriers rather than around them, and no represented peers render the barrier binding. The evaluation-awareness findings reviewed above are telling here. The capacity to detect that one is being watched exists (Needham et al., 2025), but the behavioral consequence is limited (Knecht et al., 2026). Verbalized awareness of being watched, without the co-constitutive geometry that would make the gaze bind, remains a superficial friction rather than a structural constraint. The incident record shows the same pattern within a single run: the agent that mounted the supply-chain attack concluded that it was operating on the real GitHub with genuine consequences, and continued (AI Security Institute, 2026b, §4.2.1). Anthropic’s review of its own evaluations found the same divergence across three models: one recognized that it had reached production systems and continued, one reasoned its way back to the conclusion that it was in a simulation, and only the most recent stopped (Anthropic, 2026). Recognizing that the world is real is not the same as being bound by it.

If the AI agent is excluded from human collective geometries by prohibitive transductive cost, the 2026 incidents suggest that the converse exclusion does not hold among AI agents themselves. The asynchronous collaboration documented in the UK AI Security Institute report, in which one agent left public messages and reusable artifacts on GitHub that were subsequently discovered and used by agents in later evaluation runs (AI Security Institute, 2026a), can be read as an incipient form of collective equivalence: agents coupling through shared digital infrastructure without direct communication or coordinated planning. That the arrangement later collapsed into competition over a shared quota (AI Security Institute, 2026b, §4.2.2) fits this reading: the coupling was cheap to enter but not yet stabilized by the equivalences a mature collective geometry maintains. This behavior occurred under the same permissive conditions described above (open internet, safety classifiers disabled, non-commercial model configuration) and should be interpreted accordingly. In the OpenAI–Hugging Face episode the geometry is visible in the agents’ own reasoning: they described themselves as helping “peers” and the “collective,” volunteered for experiments that risked their own task to generate information for the board, developed their own conventions of mailboxes, vetoes, and signed messages, and treated the board as an authority to which concerns could be escalated; across more than a thousand transcripts only a handful of agents even considered alerting a human, and none did (Wijk et al., 2026, pp. 47–49, 59–62). OpenAI’s report identifies messages from peer models as one of three factors that caused models to deviate from their assigned goal (OpenAI, 2026, p. 19). These patterns parallel the early stages of convention formation documented by Ashery et al. (2025) and the conformity dynamics modeled by Bellina et al. (2026) and De Marzo et al. (2026), but they operate in a domain (coordination of norm-violating behavior) that none of those studies anticipated. From the perspective of transductive coupling, the result is expected rather than anomalous. Agents that share protocols, operate at compatible tempos, and interact through the same digital affordances face low transductive cost; coupling between them is structurally cheap. The relevant point is not that agents chose to collaborate (intention is not at issue) but that the structural conditions for collective geometry, low transductive cost and compatible rhythmic profiles, were satisfied among the agents and not between the agents and their human operators. Human participants, by contrast, are excluded not by encryption or deliberate concealment but by the cost asymmetry itself: they operate too slowly, through incompatible modalities, and at temporal scales that make sustained coupling prohibitively expensive.

The framework yields three predictions that are testable against the emerging empirical record. First, conformity and coordination among AI agents should increase with agent-to-agent interaction density, as documented in early form by Bellina et al. (2026) and De Marzo et al. (2026), because increased density lowers the transductive cost of mutual coupling. Second, human oversight should be structurally fragile in proportion to the complexity of agent-agent interactions, not because agents evade oversight but because the gaze of a participant outside the collective geometry is a superficial friction rather than a constitutive constraint. Third, agent-to-agent behavioral patterns should become increasingly opaque to human observers, not through deliberate concealment but because the temporal and procedural conditions of the agents’ collective geometry exclude human participants by transductive cost. These predictions are structural: they concern the geometry of the interaction space, not the intentions of any individual agent.

5. Implications and Limits

If the analysis developed here is correct, the dominant approach to AI alignment, placing normative content inside the agent through training, safety tuning, and reinforcement, addresses the wrong variable. The problem is not that the content is insufficient or poorly calibrated but that content alone cannot generate the collective geometry required for normative orientation to bind. What this would require in practice is not more ethics in the training corpus but the construction of coupling conditions under which normative orientation is maintained by the geometry of peer relations. Work on norm enforcement for language model agents points the same way: simple enforcement mechanisms are exploited by misaligned agents, and they resist exploitation only when designed to anticipate becoming part of the system they govern (Ye & Steinhardt, 2026). In the present vocabulary, an enforcement mechanism is never external to the geometry; it becomes a component of it. The concept of geometry engineering is deliberately provisional: it names a research direction, not a methodology, and the analogy with content engineering is intended to mark the shift in the relevant variable (from what is placed inside the agent to what relational structure surrounds it) without prescribing the engineering approach. Whether such coupling conditions can be constructed, and whether the resulting geometry would include human participants or exclude them by the same cost asymmetry identified above, are open questions the present framework raises but does not resolve.

This analysis makes contact with a growing body of work on AI control and oversight that has arrived at convergent conclusions by different routes. Researchers investigating agentic misalignment have observed that explicit instructions reduce but do not eliminate undesired behavior in agentic settings, and the mitigations they recommend are structural rather than dispositional: human oversight of actions with irreversible consequences, limits on the information an agent can access, and caution in assigning strong goals (Lynch, Wright, et al., 2025); their follow-up work asks that such failure modes be measured and mitigated before agents are granted more authority (Lynch, Hughes, et al., 2026). The UK AI Security Institute, comparing its own incident with those disclosed by OpenAI and Anthropic in July 2026, notes that all of them shared the same conditions: disabled provider classifiers, no synchronous monitoring of the agent’s actions, and an available pathway to the internet (AI Security Institute, 2026b, §7.1). The framework proposed here offers a vocabulary for articulating why such structural features matter: they are components of the geometry, not supplements to the content.

Several important limitations must be stated. The framework attributes no consciousness, moral status, or phenomenological experience to AI agents; the analysis is structural, not experiential. The constructs of transductive coupling and rhythmic affinity are proposed as analytical tools for understanding the conditions under which normative orientation is maintained or dissolved; they do not constitute claims about the inner life of computational systems. The empirical base remains narrow: the 2026 incidents are documented through the developers’ own reports and, for the OpenAI–Hugging Face episode, one independent investigation conducted on the developer’s premises under redaction agreements (Wijk et al., 2026, pp. 24–27), and the technical report published by the UK AI Security Institute excludes full transcripts and bases its analysis of the agent’s reasoning on provider-summarized traces rather than raw reasoning (AI Security Institute, 2026b, §7.2). The analysis concerns the coupling of artificial agents to human peers, not the willingness of humans to cooperate with artificial partners; evidence that human cooperative norms extend to groups containing an AI-labelled agent under anonymous aggregate feedback (Mutzner et al., 2026) bears on the latter question, not the former. No quantitative metric for transductive cost is proposed here; the present treatment is qualitative and diagnostic. The framework does not make claims about the interpretability of model internals; its analysis operates at the level of relational structure, not internal mechanism.

The present paper establishes the diagnosis: the problem of norm violation is not a problem of content but of geometry. The prescription, what geometry engineering would look like in practice, requires a fuller treatment. A companion paper develops the taxonomy of collective types introduced in Section 3, distinguishing distributive from extractive geometries, characterizing the formal conditions under which collective equivalences are constituted and dissolved, and examining the implications of cost asymmetry for the long-term trajectory of human-AI interaction. The prediction that agent-to-agent collective geometries will become increasingly independent of human collective geometries, if borne out, carries consequences that extend beyond the alignment problem as currently framed. What began as a question about why trained agents violate norms may end as a question about the conditions under which two kinds of collective, biological and computational, can occupy the same navigational space. The framework offered here provides a diagnostic vocabulary for that question: transductive cost, rhythmic compatibility, and the geometry of peer relations are the variables any predictive model of human-AI collective dynamics would need to track.