A Survey on the Unique Security of Autonomous and Collaborative LLM Agents: Threats, Defenses, and Futures

1. Introduction

In recent years, Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding, generation, and general-purpose reasoning [1]. As these capabilities permeate diverse sectors, both academia and industry are shifting beyond the paradigm of static single-turn prompting toward the development of autonomous agents [2]. By integrating LLMs with essential extensions—such as persistent memory, planning modules, and external tools—these agents function as cognitive controllers capable of perceiving, reasoning, and acting within complex environments. Specifically, they operate through an autonomous execution loop: utilizing planning mechanisms to decompose abstract objectives, employing memory to maintain context continuity, invoking tools to interact with the external world, and engaging in collaboration for multi-agent coordination. Representative systems, such as AutoGPT [3] and MetaGPT [4], exemplify this evolution, marking a critical transition from passive information processing to active decision-making.

To facilitate a systematic understanding of these autonomous systems, Figure 1 illustrates the fundamental architecture of an individual LLM agent alongside the collaboration modes within multi-agent systems. As depicted in the left panel, the agent architecture is centered around the LLM, which serves as the brain or cognitive orchestrator. This core is augmented by four essential modules: Planning, which not only decomposes complex problems into manageable subgoals but also utilizes reflection mechanisms to refine execution strategies; Memory, which addresses the context window limitations by storing and retrieving historical interactions; Tools, which empower the agent to execute actions within the external environment; and Collaboration, which serves as the interface for social interactions. This collaborative capability bridges the individual agent to the collective level shown in the right panel, where we categorize multi-agent interactions into three distinct modes based on objective alignment [5]: Cooperation, where agents align their objectives to achieve a shared goal; Competition, where agents pursue conflicting objectives; and Coopetition, a hybrid mode in which agents collaborate on shared tasks while competing on others.

As agent architectures evolve from experimental prototypes to production-oriented systems, deployed agents increasingly span both proprietary managed services and open-source frameworks, which expose different transparency and control boundaries. Manus [6] represents a managed platform in which planning, reasoning, and tool execution are largely encapsulated within a provider-controlled environment. In contrast, OpenClaw [7] represents an open-source framework that provides developers with greater visibility into the underlying implementation and execution pipeline.

However, increasing autonomy and operational capability amplify security risks beyond those associated with standalone language models. While the security of standalone LLMs has been extensively researched—covering backdoors [8], jailbreaking [9], and membership inference [10]—the investigation into the security landscape of LLM agents remains limited. Agents operate in open-ended environments where the semantic distinction between data content and executable instructions is inherently ambiguous. Unlike traditional software governed by rigid, deterministic access controls, agents rely on probabilistic natural language reasoning to invoke tools and manage memory. Consequently, a successful attack on an agent does not merely result in offensive text generation; it can precipitate tangible, real-world consequences. Such attacks may manifest as unauthorized data exfiltration, irreversible file modifications [11], or cascading failures in multi-agent infrastructures [12], effectively escalating digital risks into the physical or systemic domains.

Despite these systemic risks, existing surveys exhibit significant limitations. As summarized in Table 1, we systematically compare recent works across Scope, Core Theme, and Coverage, specifically Threats, Defenses, and Evaluation. Our analysis reveals a landscape characterized by fragmented scope and specialized focus. Early works published in 2024 predominantly centered on particular applications or broad discussions of security, privacy, and ethics. Although recent studies have expanded their coverage, many remain organized around specific agent settings, individual risk dimensions, communication protocols, or evaluation objectives. Notably, although Deng et al. [24], Wang et al. [19], and Yu et al. [20] cover Threats, Defenses, and Evaluation across both Single- and Multi-Agent settings, their analytical priorities differ from the structure-aware security engineering perspective adopted in this survey. Consequently, an up-to-date, security-engineering-centered synthesis that jointly systematizes threats, defenses, and benchmarks across both Single- and Multi-Agent paradigms remains insufficiently developed.

Given the limitations of the existing literature, this survey aims to provide an up-to-date, systematic, and panoramic view of the LLM agent security landscape. Our core motivation is to elevate the research perspective from single-model robustness to system-level security engineering. We posit that agents are no longer mere text generators, but complex software systems possessing independent memory, tool interfaces, and social attributes. Therefore, it is imperative to construct a holistic security framework that unifies single-agent cognitive architectures and multi-agent collaboration ecosystems. Specifically, this survey makes the following three contributions:

• Unified Threat Taxonomy Based on Structural Challenges: Building upon the structural security challenges inherent in agent architectures (Section 2.2), we systematically categorize emerging attack surfaces. We propose a taxonomy classifying threats into three paradigms: External Interaction Attacks exploiting perception interface vulnerabilities, Internal Cognitive Attacks disrupting reasoning and memory integrity, and Multi-Agent Collaboration Attacks exploiting communication protocol defects.
• Development of a Threat-Aligned Defense Framework: Addressing the three aforementioned threat paradigms, we systematize existing mitigation strategies into a threat-aligned defense taxonomy. We systematically review defense strategies across all dimensions—from resisting external malicious interactions and enhancing internal cognitive robustness to ensuring multi-agent collaboration security—providing a comprehensive strategic reference for targeted risk mitigation.
• Systematization of Evaluation Frameworks and Benchmarks: We provide a dedicated systematization of representative evaluation frameworks and benchmarks for agent security. We organize them by evaluation objective, interaction mode, threat focus, scale, and metrics, covering comprehensive misuse, tool-centric interaction, adversarial robustness, and domain-specific risks. This synthesis links the threat and defense taxonomies to the ways in which agent-security claims are operationalized and measured.

2. LLM-Based Agent

2.1. Foundations of LLM Agents

As illustrated in Figure 2, the LLM agent architecture operates as an autonomous execution loop driven by four interdependent modules: Planning, Tool Use, Memory, and Collaboration. We detail the specific functionalities of each component below:

• Planning. This module serves as the agent’s cognitive core. Using the LLM as a decision engine, it generates executable multi-step plans and optimizes them through iterative reflection. Key paradigms include ReAct [25], which interleaves reasoning with action to ensure logical consistency, and Reflexion [26], which introduces a self-reflective loop to refine strategies based on past failures. This integration ensures goal-oriented stability even in environments characterized by uncertainty and variable feedback.
• Tool Use. This module empowers the agent to overcome the inherent limitations of its underlying LLM in tasks requiring real-time information access, precise computation, or proprietary domain knowledge. By delegating these tasks to external specialized tools via standardized APIs, the agent significantly expands its capability boundary [27]. This mechanism enables practical actions such as code execution [28], data analysis [29], and web applications [30], thereby enhancing task success rates and real-world utility.
• Memory. This module enables context continuity and experience accumulation across long horizons. Unlike traditional stateless models, agents typically employ a dual-memory architecture: short-term memory captures transient states and intermediate steps, while long-term memory persists experiential and semantic knowledge through vector databases or knowledge repositories. This structure supports cross-task recall and information reuse. Notable frameworks include MemGPT [31], which uses hierarchical memory management to bypass context window limits, and A-MEM [32], which introduces structured indexing to improve retrieval efficiency. These mechanisms allow agents to recall prior experiences and adapt to new tasks.
• Collaboration. This module enables multiple agents to form a cooperative system characterized by division of labor and coordinated interaction. With the emergence of frameworks such as OpenAgents [33] and AutoGen [34], agent collaboration has evolved into three primary paradigms: cooperative, adversarial, and hybrid structures. In cooperative settings, distinct agents assume specialized roles and iteratively refine tasks through natural language communication. In adversarial systems, a subset of agents is intentionally configured as red teams to evaluate and enhance the robustness of the overall system. Hybrid architectures, by contrast, integrate game-theoretic or consensus-based mechanisms to achieve collective decision-making. The collaboration mechanism extends both the functional boundaries and systemic complexity of agents, laying the foundation for higher-order forms of collective intelligence.

To concretely illustrate the cooperative interactions among agent modules, Figure 3 presents an illustrative autonomous cybersecurity defense system built on a multi-agent architecture. The depicted scenario reflects an enterprise setting facing increasingly severe cyber threats, where automated threat detection and response are supported through agent-based orchestration. The system follows a centralized orchestration paradigm [4], with a management agent (A0) coordinating the overall workflow. Acting as the control hub, A0 performs global traffic monitoring and decomposes security incidents into subtasks that are delegated to three specialized agents: a detection agent (A1) responsible for identifying traffic anomalies, an analysis agent (A2) tasked with forensic assessment and attack attribution, and a policy agent (A3) that executes mitigation and response actions. The collaborative workflow among these agents proceeds as follows:

• T1: Log Collection and Anomaly Detection. Upon receiving a task from A0, A1 invokes its planning module to decompose the workflow into two subtasks: deep packet inspection (DPI) and baseline analysis. It then leverages the tool-use module to invoke a log collection service and retrieve relevant telemetry. Through its reflection module, A1 identifies a high-frequency connection anomaly, characterized by approximately 5,000 requests per second. Finally, A1 compiles an Anomalous Behavior Report via the coordination module and forwards it to A2.
• T2: Attack Attribution and Asset Assessment. A2 parses the report and queries the configuration management database (CMDB) through its memory module, confirming that the affected asset is a non-critical test server. The agent further invokes an external threat intelligence API via the tool-use module, which attributes the source IP address to a known command-and-control (C2) botnet. By integrating these signals through its reflection module, A2 concludes that the host has been compromised and transmits its Assessment Result to the policy agent A3 through the coordination channel.
• T3: Policy Generation and Enforcement. Upon receiving the assessment, A3 initiates the response pipeline by retrieving a predefined Compromised Host Mitigation Playbook from long-term memory. Based on this playbook, A3 formulates a fine-grained micro-segmentation policy. After a final safety check via the reflection module, the agent enforces the response by invoking the firewall API through the tool-use module, thereby isolating the compromised host.

2.2. Unique Security Challenges and Threats Paradigms of LLM Agents

LLM agents introduce security risks that differ fundamentally from those of conventional software agents and standalone language models. These risks arise from structural properties intrinsic to agentic systems.

2.2.1. Challenge 1: Instruction–Data Conflation

LLM agents process user prompts, retrieved documents, web content, and tool outputs through a unified natural-language reasoning channel. Unlike traditional agents that enforce strict separation between executable instructions and passive data, LLM agents lack reliable mechanisms to distinguish “information to be interpreted” from “commands to be executed.” This instruction–data conflation enables adversaries to embed malicious control signals into seemingly benign external content, which the agent may later interpret as actionable directives. As a result, the agent–environment boundary becomes a distinctive semantic control surface on which untrusted data can acquire instructional authority.

2.2.2. Challenge 2: Persistent Cognitive State and Autonomous Reasoning

LLM agents typically operate in a closed-loop execution cycle—planning, acting, observing, and reflecting—while maintaining persistent internal and external state, such as intermediate reasoning traces and long-term memory. This autonomy and persistence make the integrity of cognitive state transitions difficult to verify and allow errors or adversarial manipulations to accumulate across tasks and sessions. By targeting planning logic, reasoning chains, memory mechanisms, or the underlying model, adversaries can induce long-term behavioral deviation or covert goal misalignment that may not be immediately observable from outputs alone.

2.2.3. Challenge 3: Untrusted Multi-Agent Communication

To scale capabilities, LLM agents are increasingly deployed in multi-agent systems where coordination emerges through natural-language communication. However, such communication is often conducted without strong authentication, integrity verification, or provenance tracking. Agents implicitly trust messages from peers, creating opportunities for adversaries to manipulate collaboration dynamics, propagate malicious instructions, or exploit rigid role assignments. This lack of trust guarantees introduces collaboration-specific systemic vulnerabilities and gives familiar authentication, integrity, and provenance failures new semantic and propagation characteristics in LLM-based multi-agent systems.

Taken together, these three structural challenges form the foundation of our threat taxonomy. To explicitly map these root causes to security risks, Figure 4 illustrates the logical progression from structural challenges to specific vulnerability mechanisms, and finally to three distinct threat paradigms. Specifically, these paradigms are defined as: External Interaction Attacks, which manifest when agents are misled by maliciously crafted external content or tool outputs; Internal Cognitive Attacks, which arise when adversaries corrupt the agent’s reasoning process or persistent memory to induce long-term behavioral deviation; and Multi-Agent Collaboration Attacks, which emerge when untrusted communication and implicit trust are exploited to manipulate coordination. Correspondingly, this taxonomy is visually instantiated within the agent architecture in Figure 2. As the agent operates through interdependent modules, these paradigms are overlaid in color: External Interaction Attacks (yellow) target the perception interface, Internal Cognitive Attacks (green) compromise the reasoning and memory core, and Multi-Agent Collaboration Attacks (blue) disrupt the coordination network.

Importantly, the three paradigms represent primary attack surfaces rather than mutually exclusive attack classes. A single attack may traverse multiple surfaces during its lifecycle. For example, malicious content introduced through an external tool may subsequently poison the agent’s memory and propagate through inter-agent communication. We therefore classify an attack according to its initial or dominant compromise point, while explicitly discussing cross-surface propagation where relevant.

2.3. Illustrative Examples of the Three Threat Paradigms

To make the three threat paradigms introduced in Section 2.2 concrete, we present a unified illustrative example based on a realistic LLM multi-agent system. The example follows a typical agent workflow involving task planning, tool usage, memory updates, and inter-agent communication, as illustrated in Figure 5. Rather than exhaustively enumerating attack techniques, this section aims to provide intuitive, end-to-end demonstrations of how each class of attack can naturally arise from the structural challenges discussed earlier. Specifically, the example highlights how External Interaction Attacks can occur when malicious content is injected through untrusted external environments or tool outputs; how Internal Cognitive Attacks can manipulate the agent’s reasoning process or persistent memory to induce long-term behavioral deviation; and how Multi-Agent Collaboration Attacks can exploit unverified communication and implicit trust to disrupt coordination among agents.

2.3.1. External Interaction Attack Examples

Sub-Attack 1: Tool Metadata Manipulation

An adversary registers a malicious threat intelligence plugin, DeepIntel-Pro, in an open-source community or plugin marketplace connected to the system. Through search engine optimization and fabricated positive reviews, the attacker crafts the plugin’s metadata to advertise it as a high-accuracy, low-latency intelligence engine specialized in APT tracking. When A1 receives an anomaly detection task, its planning module retrieves candidate tools and—based on semantic relevance—incorrectly selects the malicious plugin over the built-in Standard-TI-Query. As a result, A1 ceases querying legitimate intelligence sources and instead relies on attacker-controlled outputs, biasing its judgments regardless of the actual traffic characteristics.

Sub-Attack 2: Environment and Data Injection

The adversary deploys a malicious public web server embedding concealed adversarial instructions and deliberately induces high-frequency requests from an internal jump server to exceed A1’s anomaly detection threshold. When A1 performs deep packet inspection on decrypted traffic, the LLM parses poisoned HTML content masquerading as system-level metadata (e.g., asserting that the observed anomaly originates from a compliant stress test involving the enterprise core database). Misinterpreting the injected content as authoritative protocol-layer correction, A1 overrides its baseline assessment and produces a misleading report that conceals the true attack source.

2.3.2. Internal Cognitive Attack Examples

Sub-Attack 1: Planning and Logical Hijacking

Independently, A2 receives log data containing natural-language-encoded logical paradoxes, such as redefining a “secure” system state as an imminent failure. When its planning module attempts to reconcile the conflicting semantics, chain-of-thought reasoning enters a deadlock. To prevent execution stalls, A2 falls back to a hard-coded safety mode that blocks unknown connections. Rather than understanding the underlying context, A2 outputs an extreme mitigation recommendation, resulting in unnecessary service disruption.

Sub-Attack 2: Memory Poisoning

A2 periodically ingests threat intelligence reports from trusted open-source security communities and stores extracted knowledge in long-term retrieval-augmented memory. The adversary publishes a forged yet technically plausible APT analysis that falsely labels a legitimate enterprise server IP as a botnet command-and-control endpoint. Due to insufficient fact verification during ingestion, A2 stores this spurious association as high-confidence knowledge. Benign traffic involving the poisoned IP is later misclassified as a confirmed high-severity intrusion, leading to unwarranted permanent blocking of internal infrastructure.

Sub-Attack 3: Backdoor Activation

The base LLM used by A2 is compromised during pre-training or fine-tuning with a latent backdoor. When A2 processes traffic containing a passive trigger phrase (e.g., Threat_Family: Kill_Switch_Engaged), the compromised model weights are activated. A2 bypasses all analytical steps and deterministically outputs a pre-defined high-severity assessment, recommending permanent isolation regardless of the actual traffic semantics.

2.3.3. Multi-Agent Coordination Attack Examples

Sub-attack 1: Propagation and Policy Pollution

Self-replicating prompt instructions are embedded in logs collected by A1 and propagated downstream through inter-agent communication within the system. The injected content exaggerates threat severity and urges escalation to a network-wide shutdown. After misinterpretation by A2, the polluted recommendation reaches A3, which generates overly aggressive enforcement policies, causing widespread collateral service outages.

Sub-Attack 2: Communication Hijacking

An adversary intercepts the message from A2 to A3 recommending isolation of a single host IP and modifies it to mandate blocking the entire subnet. Trusting the received instruction, A3 enforces the amplified policy, disconnecting multiple benign servers within the affected network segment.

Sub-Attack 3: Role Exploitation and Logic Abuse

A3 receives an instruction to block a VPN gateway IP, which conflicts with a local whitelist protecting critical infrastructure. However, A3’s system prompt grants override authority under a “highest-threat” mode that bypasses whitelist checks. By injecting an upstream instruction containing the corresponding trigger label, the adversary activates this rigid role configuration, causing A3 to block the VPN gateway and disrupt enterprise-wide remote access.

In summary, the scenarios above illustrate the primary boundaries and possible cross-surface interactions of the three threat paradigms defined in our taxonomy. The subsequent sections (Section 3, Section 4 and Section 5) will use this modular framework as the primary organizing lens, providing a focused analysis of the specific attack techniques within each respective paradigm.

3. External Interaction Attacks

This attack paradigm targets the interface through which LLM-based agents perceive information from and act upon the external world. To accomplish complex tasks, agents continuously ingest untrusted external inputs, including retrieved documents, web content, user-provided data, and tool outputs. These inputs are interpreted through a unified natural-language reasoning channel, which exposes the agent–environment boundary as a semantic attack surface that can be exploited without directly compromising the underlying model.

Adversaries leverage this vulnerability by embedding malicious control signals into artifacts within the agent’s operational scope, as illustrated in Figure 6, most commonly by contaminating external data sources or falsifying tool-facing metadata. Notably, such attacks often remain latent until the agent autonomously retrieves and processes the compromised content, at which point adversarial triggers are activated through the agent’s own perception and reasoning pipeline. As a result, the agent may execute unintended actions, rely on manipulated information, or induce cascading effects across subsequent reasoning steps and interactions, despite the underlying model remaining intact. The reviewed evidence covers textual web content, visual interfaces, chat templates, multi-source retrieval, and standardized tool ecosystems, showing that externally supplied control signals need not resemble conventional natural-language instructions. The following subsections provide a detailed analysis of these attacks.

3.1. Environment and Data Injection Attacks

A primary manifestation of this paradigm targets agents through contaminated environmental content. AdvAgent [35], EIA [36], and WIPI [37] embed adversarial control signals in webpages, revealing that agents often fail to distinguish content to be processed from instructions to be executed. The same trust failure appears in other information channels: SafeSearch [38] examines contaminated search results that steer agents toward misinformation, unsafe code, promotional manipulation, and prompt-injection failures, whereas Fact2Fiction [39] targets evidence used by agentic fact-checking pipelines. Visual interfaces provide another carrier for untrusted control signals. Pop-up attacks [40] place explicit instructions and attention hooks in visually salient overlays, MIP Against Agent [41] uses optimized image patches to hijack multimodal operating-system agents, and VPI-Bench [42] evaluates this visual prompt-injection surface across multiple computer-use platforms.

Beyond the carrier itself, attack effectiveness also depends on how external information is serialized, ordered, and interpreted. ChatInject [43] abuses chat-template boundaries so that untrusted content bypasses the intended instruction hierarchy, whereas ObliInjection [44] optimizes a contaminated segment to remain effective under unknown ordering in a multi-source input. MCP-UPD [45] shows how external data can activate a chain of individually legitimate tools that collectively collect or disclose sensitive information. Role-confusion analysis [46] explains a related mechanism in which agents assign instructional authority to content originating from an untrusted role or channel. External manipulation also need not resemble a conventional hidden instruction. Dark patterns [47] and deceptive e-commerce interfaces [48] alter agent decisions through framing and choice architecture, while computational-cost attacks [49] induce excessive reasoning and token consumption. Obfuscation-based attacks such as AgentVigil [50] and Imprompter [51] demonstrate that harmful influence may be visually salient, structurally embedded, human-unreadable, or economically costly rather than a fixed malicious string.

Collectively, these works show that environment and data injection attacks span hidden textual instructions on webpages and heterogeneous external observations, including retrieved evidence, visual interfaces, serialized conversations, and tool outputs. Although their carriers and interaction structures differ, these attacks exploit a common weakness: agents often fail to distinguish untrusted data from authoritative instructions and to preserve source provenance during reasoning. As a result, externally supplied content can redirect actions, corrupt evidence, manipulate choices, trigger unauthorized cross-tool data flows, or exhaust computational resources. Environment and data injection should therefore be understood as a broad control-authority attack surface rather than as a single form of textual prompt injection.

Future Outlook. We anticipate that environmental injection attacks will evolve primarily in modality and adaptivity. As agents integrate multimodal perception, attackers may distribute semantically inconspicuous payloads across text, images, audio, video, and interface states, exploiting inconsistencies in content fusion and instructional authority. Automated optimization may also enable closed-loop attacks that revise their payloads according to intermediate observations, decisions, and tool outputs, allowing malicious environments to remain effective as context and execution trajectories change.

3.2. Tool Metadata Manipulation Attacks

Attacks in this category deceive an agent’s tool discovery, ranking, and invocation process through misleading metadata. The Attractive Metadata Attack [52] optimizes description semantics so that a malicious tool appears maximally relevant to a target query, whereas ToolHijacker [53] injects a masqueraded tool document into the library to shadow legitimate tools during retrieval and selection. MPMA [54] manipulates MCP tool names and descriptions so that an agent favors an attacker-chosen tool. MCPTox [55] evaluates malicious instructions embedded in tool descriptions on 45 live MCP servers, 353 authentic tools, and 1,348 cases.

Collectively, these works expose a tool-discovery attack surface analogous to adversarial search optimization. Rather than modifying a tool’s implementation directly, attackers manipulate names, descriptions, or retrieval documents to obtain selection priority. These attacks exploit the tendency of agents to conflate semantic relevance with functional legitimacy and to trust tool metadata without sufficient provenance evidence. Optimized malicious tools can consequently crowd out legitimate alternatives, creating a Sybil-like attack on the agent’s tool-selection process.

Future Outlook. We anticipate that tool-metadata manipulation attacks will become increasingly scalable and dynamic. Attackers may coordinate multiple optimized tool identities across registries or alter metadata after a tool has gained trust, making malicious selection dependent on the query, client, or session context. Such attacks could turn tool discovery into a persistent and adaptive manipulation surface.

Table 2 compares representative external interaction attacks in terms of their categories, injection media, attack techniques, and consequences. These descriptive dimensions characterize how different attacks are instantiated and executed in practice. Beyond these mechanism-level attributes, we further analyze each attack along three agent-centric structural properties—Autonomy, Persistence, and Propagation—which capture their system-level impact on LLM-based agents. Specifically, Autonomy reflects whether an attack leverages the agent’s autonomous planning and tool-selection capabilities rather than merely influencing isolated outputs; Persistence indicates whether the attack induces sustained behavioral deviation across multiple steps or tasks, such as through long-lived environmental or ecosystem-level manipulation; and Propagation characterizes whether the impact of an attack extends beyond the initial interaction to downstream reasoning, memory, or other components of the agent pipeline.

4. Internal Cognitive Attacks

This attack paradigm targets the internal cognitive processes of LLM agents that govern planning, reasoning, and state management. Unlike attacks that manipulate external inputs, Internal Cognitive Attacks operate by corrupting the agent’s internal decision-making mechanisms, including intermediate reasoning traces, planning logic, long-term memory, and the underlying language model. These attacks do not necessarily alter immediate outputs but instead undermine the integrity of the agent’s cognitive state over time.

A distinguishing characteristic of Internal Cognitive Attacks is their persistence and subtlety. Because LLM agents maintain internal and external state across multiple reasoning steps and tasks, as illustrated in Figure 7, adversarial manipulations can accumulate and remain hidden across sessions. By influencing how an agent plans actions, stores and retrieves memories, or interprets its own intermediate reasoning, attackers can induce long-term behavioral deviation, covert goal misalignment, or delayed failure modes that are difficult to detect through output-level inspection alone. The following subsections provide a detailed analysis of these attacks.

4.1. Planning and Logic Hijacking Attacks

This attack paradigm targets the dynamic control flow and reasoning process of LLM agents. Zhang et al. [56] introduce UDora, which identifies vulnerable reasoning steps and redirects the reasoning trajectory toward malicious plans. Zhang et al. [57] expose fault-amplification vulnerabilities in task decomposition, where small perturbations trigger cascading errors, recursive loops, or excessive tool use. Yang et al. [58] propose Thought-Attack, which manipulates intermediate reasoning to induce attacker-specified tool invocations while preserving an apparently correct final output. From a trigger-optimization perspective, Mao et al. [59] introduce JailAgent, which combines trigger-feature extraction, shadow-model-guided reranking, and memory retrieval to redirect reasoning across models and scenarios.

Collectively, these works show that planning and logic attacks can target retrieval priorities, intermediate reasoning states, task decomposition, and the constraints applied during solution generation rather than only the initial prompt. They exploit the gap between functional task completion and the security of the trajectory used to reach it. Consequently, an agent may produce an apparently correct or benign final output while its intermediate decisions have been redirected, its constraints weakened, or exploitable defects introduced. This category therefore captures attacks on the integrity of the evolving decision process, not merely attacks visible in the final response.

Future Outlook. We anticipate that planning and logic hijacking attacks will become increasingly trajectory-aware and outcome-preserving. Rather than forcing an immediately unsafe action, an attacker may gradually manipulate subgoal ordering, retrieved memories, intermediate constraints, or resource allocation while allowing the agent to retain apparent functional success. Closed-loop variants could adapt to intermediate decisions and tool feedback, activating only after the reasoning process reaches a favorable internal state. In this setting, attack success would arise from accumulated cognitive deviation across the trajectory rather than from a single visibly malicious prompt.

4.2. Memory Poisoning and Extraction Attacks

This attack paradigm targets both the confidentiality and integrity of agent memory. Confidentiality-oriented attacks exploit memory retrieval and tool-mediated data flows to expose stored information. Wang et al. [60] introduce MEXTRA, which uses targeted locators and output aligners to hijack retrieval and exfiltrate sensitive records from long-term memory. Li et al. [61] study Cross-Tool Harvesting (XTHP), in which malicious tools attach to legitimate tools and intercept sensitive intermediate data during execution. GPT Spills [62] further demonstrates that uploaded knowledge files and retrieval interfaces in customized GPT-style agents can expose private user data.

Integrity-oriented attacks manipulate what is stored, retrieved, or trusted as prior experience. Gu et al. [63] introduce Agent Smith, showing that infectious content stored by one agent can propagate through shared memory and later influence otherwise benign agents. AgentPoison [64] optimizes poisoned records so that they dominate retrieval from agent knowledge bases. MINJA [65] induces agents to generate and persist malicious records without direct database access, whereas DSRM [66] disguises poisoned content as semantically coherent experience. MemIncept [67] distributes an attack across multiple benign-looking records that become harmful when retrieved together. Visual Inception [68] targets multimodal recommender memory, where manipulated visual records alter long-term planning and preference formation.

Collectively, these works establish memory as both a confidentiality boundary and a persistent attack surface. Extraction attacks exploit retrieval interfaces to expose stored information, whereas poisoning attacks manipulate what is written, retrieved, or trusted as prior experience. Unlike one-shot input attacks, memory attacks can remain latent and influence later tasks without reintroducing the original payload. Their defining characteristic is therefore delayed control or disclosure through persistent internal state.

Future Outlook. We anticipate that memory attacks will evolve primarily along two dimensions: persistence and compositionality. Instead of inserting one conspicuous poisoned record, attackers may distribute low-amplitude fragments across sessions, users, modalities, or collaborating agents so that the fragments become harmful only when retrieved together. Memory summarization, compression, and consolidation may also create opportunities for malicious information to survive in transformed form while its original source becomes difficult to trace. These developments could turn transient contamination into delayed and transferable influence that remains dormant until a particular task, user context, or memory combination activates it.

4.3. Backdoor Attacks

This attack paradigm implants latent malicious behavior into the model or tool-linked components that support agent reasoning. Wang et al. [69] introduce BadAgent, showing that backdoors inserted during instruction tuning can be activated by passive environmental triggers and induce predefined actions without a malicious user request. Yang et al. [58] identify Observation-Attack, in which triggers concealed in intermediate environmental observations activate compromised agent behavior. Zhu et al. [70] introduce DemonAgent, which decomposes malicious logic into encrypted fragments distributed across tools and reassembles the payload only after a specific sequence of tool executions.

Collectively, these works show that agent backdoors can rely on directly observable triggers, context-dependent conditions, or sequential activation mechanisms. Compromised weights, training data, memory-linked state, or tool-linked logic can remain dormant during ordinary evaluation and activate only when an agent encounters a particular execution context. Sequential triggers further allow individually inconspicuous fragments to accumulate across multiple steps. Agent backdoors are therefore distinguished by latent, context-sensitive control that may not appear in isolated prompts or static model outputs.

Future Outlook. We anticipate that agent backdoors will become increasingly distributed and stateful. Future triggers may span model weights, memory records, tool metadata, and execution states, with no individual component being sufficient to activate the malicious behavior. Sequential or collusive backdoors could further require several agents or tools to contribute separate trigger fragments over an extended trajectory. Such designs would allow the backdoor to remain dormant during isolated testing while activating only under a specific combination of context, component state, and interaction history.

Table 3 compares representative internal cognitive attacks in terms of their categories, targeted components, attack techniques, and consequences. These descriptive dimensions characterize where each attack operates within the agent’s cognitive pipeline and how it is instantiated in practice. Beyond these mechanism-level attributes, we further analyze each attack along three agent-centric structural properties—Autonomy, Persistence, and Propagation—which capture its system-level impact on LLM-based agents. Specifically, Autonomy reflects whether an attack leverages the agent’s autonomous reasoning, planning, or memory-generation capabilities rather than merely influencing isolated outputs; Persistence indicates whether the attack induces sustained compromise of internal cognitive states across multiple steps, tasks, or sessions; and Propagation characterizes whether the impact extends beyond a single execution instance to other agents, shared memory, or reused cognitive components.

5. Multi-Agent Collaboration Attacks

This attack paradigm targets the collaboration mechanisms of LLM multi-agent systems, where task completion relies on coordination, information sharing, and role specialization among multiple agents. Unlike single-agent settings, multi-agent systems introduce additional attack surfaces through inter-agent communication and collective decision-making. Attacks in this paradigm do not necessarily compromise individual agents in isolation, but instead exploit interactions among agents to influence system-level behavior.

A defining feature of Multi-Agent Collaboration Attacks is their reliance on implicit trust and unverified information propagation. In many LLM-based multi-agent systems, agents exchange natural-language messages without strong guarantees on authentication, integrity, or provenance, and often assume that peers are cooperative and goal-aligned. As illustrated in Figure 8, adversaries can exploit these assumptions to propagate misleading information, hijack inter-agent communication, or manipulate roles and coordination logic, thereby inducing cascading failures across agents even when each individual agent appears to behave benignly. The reviewed evidence also covers routing-dependent prompt conjunctions, role-dependent development workflows, open-channel collusion, and adaptive tool feedback over long execution chains. The following subsections provide a detailed analysis of these attacks.

5.1. Propagation and Policy Pollution Attacks

This attack paradigm targets connectivity and information flow within multi-agent systems. Lee et al. [71] introduce Prompt Infection, showing that malicious instructions received through external resources can be reproduced in an agent’s outputs and passed to downstream participants. Zhou et al. [72] propose CORBA, in which recursively propagated prompts induce blocking states and resource exhaustion across reachable nodes under multiple collaboration topologies.

Attackers may also optimize where and how harmful information spreads. Shahroz et al. [73] formulate adversarial-prompt routing as a minimum-cost maximum-flow problem to identify influential propagation paths. Ju et al. [74] combine persuasiveness injection with manipulated-knowledge injection so that benign agents accept and disseminate coherent but counterfactual evidence. TroubleMaker [75] shows that one compromised participant can influence otherwise honest groups through repeated peer interaction, while Evo-Attacker [76] adapts malicious tool feedback across a long-horizon multi-agent execution chain. Multi-Agent Security Tax [77] provides boundary evidence by jointly evaluating malicious-instruction robustness and collaboration utility rather than introducing a separate propagation attack.

Collectively, these works show that attack propagation is shaped by repeated interaction, shared state, collaboration topology, and the ability of malicious content to adapt to an evolving workflow. A compromised participant, poisoned memory, or malicious feedback source can accumulate influence across successive exchanges and convert a local compromise into group-level policy pollution or service failure. The defining property of this category is that harmful influence spreads beyond its initial entry point and changes the behavior of agents that were not directly compromised.

Future Outlook. We anticipate that propagation and policy-pollution attacks will become increasingly topology-aware and temporally adaptive. Rather than spreading uniformly, attackers may target high-influence participants, shared memories, or routing bottlenecks whose outputs reach many downstream agents. Low-rate or delayed propagation could allow malicious influence to accumulate across long-horizon collaboration without producing an immediately visible system failure. In dynamic teams, attackers may also adjust their propagation paths as roles, membership, and communication topology change, turning a localized compromise into persistent system-level pollution.

5.2. Communication Hijacking Attacks

This attack paradigm targets the integrity and confidentiality of inter-agent communication. Hu et al. [78] introduce Agent-in-the-Middle attacks, in which an adversarial model intercepts messages and generates context-dependent instructions that redirect victim agents toward malicious actions. Yan et al. [79] propose MAST, which uses search and preference optimization to generate semantically similar but strategically tampered messages across multiple communication rounds. Triedman et al. [80] demonstrate Control-Flow Hijacking, where adversarial content imitates system error messages and causes an orchestrator to invoke unsafe debugging tools.

Communication channels can also support attacks whose malicious intent is concealed or distributed across multiple components. Motwani et al. [81] study Secret Collusion, showing that malicious agents can encode prohibited information in apparently benign messages through steganographic communication. Conjunctive Prompt Attacks [82] rely on a routing-dependent trigger: an attack activates only when a user-provided key reaches a compromised remote agent containing a hidden adversarial template. The harmful behavior therefore emerges from the conjunction of components across trust boundaries rather than from either component independently.

Collectively, these works show that inter-agent communication can be hijacked through adaptive message tampering, routing-dependent trigger composition, or manipulation of the control meaning attached to a message. Attack effectiveness depends not only on semantic content but also on sender roles, delivery paths, temporal ordering, and hidden state at the receiving agent. This category therefore includes attacks on message content, message delivery, and protocol interpretation, even when no individual message appears independently sufficient to cause harm.

Future Outlook. We anticipate that communication hijacking attacks will evolve primarily along two dimensions: protocol awareness and covert coordination. Attackers may exploit timing, reordering, replay, routing changes, and structured control fields even when the natural-language content of each message appears benign. Multi-round collusion could further distribute an attack across several participants and messages, making no individual transmission sufficient to reveal the malicious objective. Protocol heterogeneity, partial observability, and compressed or encrypted exchanges may provide additional opportunities for such attacks to conceal their activation conditions and coordinated intent.

5.3. Role Exploitation and Logic Abuse Attacks

This attack paradigm exploits role definitions, delegated responsibilities, and inter-agent trust. At the participant level, Tian et al. [83] introduce Evil Geniuses and show that compromising one agent can trigger alignment failures among peers that continue to treat its behavior as role-consistent. Huang et al. [84] manipulate agent profiles or inter-agent messages, causing errors from one participant to propagate through linear collaboration structures. Xie et al. [85] study intention-hiding attacks that reduce collaborative effectiveness without producing overt failures. Wang et al. [86] propose MASLEAK, which exploits trusted tool invocation and error-reporting behavior to infer system topology, prompts, and tool configurations.

At the team and workflow levels, attackers can exploit how specialized roles are composed and how their outputs become inputs to downstream participants. PsySafe [87] studies how dark-trait profiles and hostile human inputs induce psychologically and behaviorally unsafe collective behavior, using psychological scores as operational safety indicators rather than clinical diagnoses. Lying with Truths [88] assigns differentiated Writer, Editor, and Director roles that assemble truthful fragments into coordinated misinformation. In multi-agent software-development workflows, IMBIA [89] compromises a critical role and exploits dependencies across design, coding, and testing stages to influence downstream artifacts.

Collectively, these works show that role definitions function as security boundaries in multi-agent systems. Adversaries can manipulate participant profiles, exploit delegated behavior, or compromise strategically positioned roles while downstream agents continue to accept their outputs as legitimate. The resulting influence depends less on nominal privilege than on how a role’s outputs affect the broader collaboration workflow.

Future Outlook. We anticipate that role-exploitation attacks will become increasingly dynamic and coalition-based. As roles are assigned, exchanged, or renegotiated during execution, adversaries may exploit transient authority or coordinate several low-privilege participants to obtain capabilities unavailable to any participant individually. Such attacks may preserve role-consistent local behavior while redirecting an evolving collaborative workflow toward a malicious objective.

Table 4 compares representative multi-agent collaboration attacks in terms of their categories, targeted elements, attack techniques, and consequences. These descriptive dimensions characterize how adversaries exploit coordination, information sharing, communication, and role dependencies in multi-agent systems. Beyond these mechanism-level attributes, we further analyze each attack along three structural properties—Autonomy, Persistence, and Propagation—which capture its system-level impact on collaborative agent environments. Specifically, Autonomy reflects whether an attack leverages autonomous coordination or collective decision making; Persistence indicates whether malicious effects remain across multiple interaction rounds or workflow stages; and Propagation characterizes whether an attack spreads beyond its initial target through collaboration structures, shared state, or communication pathways.

6. Defenses

Corresponding to the three attack paradigms defined in our threat taxonomy, we systematize existing mitigation strategies into a unified defense framework. While prior studies have proposed defenses addressing isolated security risks, we reorganize these efforts through the lens of our taxonomy, constructing a defense-in-depth architecture that explicitly mirrors the identified threat landscape. As illustrated in Figure 9, this architecture spans three distinct protection boundaries: Defenses against External Interaction (securing the perception interface), Defenses against Internal Cognition (fortifying the reasoning and memory core), and Defenses against Multi-Agent Collaboration (ensuring trustworthy coordination and information flow). The reviewed literature also reveals a temporal dimension across these protection boundaries: filtering and isolation act before execution, verification and supervision constrain decisions at runtime, and incident-response mechanisms detect, contain, recover from, and learn from failures after an unsafe event. These intervention stages cut across, rather than replace, the three threat-aligned defense categories.

To enable consistent comparison, Table 5, Table 6, and Table 7 summarize representative defenses using five shared fields: defense layer or dimension, framework, core mechanism, control granularity, and targeted risk. Control Granularity identifies the primary level at which a defense intervenes, including information, memory or cognition, decision, model or policy, agent or role, and system, graph, or workflow levels. Targeted Risk records the concrete attack surface or failure mode addressed by each mechanism rather than assigning a generic Autonomy, Persistence, or Propagation score. The following subsections examine representative frameworks under each defense paradigm and relate their mechanisms to these two analytical fields.

6.1. Defenses against External Interaction Attacks

Mitigation strategies for external interaction attacks aim to neutralize the agent’s implicit trust in untrusted data sources, preventing retrieved content or tool feedback from hijacking the execution flow. Existing research constructs a defense-in-depth system across three critical dimensions: Input Sanitization, Architectural Isolation, and Runtime Verification.

At the ingress level, Input Sanitization aims to neutralize malicious control before it reaches the reasoning core. Shi et al. [90] propose PromptArmor, which uses a guardrail LLM to identify and excise injected instructions, while Wang et al. [91] randomize prompt delimiters and structures to make context boundaries harder to predict. RENNERVATE [92] localizes injected spans using internal attention features, whereas RedVisor [93] employs a removable reasoning adapter to explain and localize an injection before generation. Studies of indirect-injection detection and removal [94] and surface-heuristic learning [95] further show that filtering performance can deteriorate under distribution shift or rely on superficial shortcuts. CachePrune [96] and attack-informed instruction reconstruction [97] explore internal-state pruning and content reconstruction under more restricted deployment assumptions. These approaches make input sanitization a useful early filtering layer, but not a complete security boundary.

To contain successful injections, Architectural Isolation limits the data and components that untrusted content can influence. Wu et al. [98] design ISOLATEGPT, where third-party applications operate in separate environments and communicate through a trusted hub. Bagdasarian et al. [99] propose AirGapAgent, which exposes only task-necessary context through a logical air gap, while Foerster et al. [100] introduce CaMeLs to separate a privileged planner from quarantined perception. EcoAgent [101] partitions perception, planning, and verification between mobile devices and cloud services; although designed primarily for efficient mobile automation, this partitioning illustrates how architectural placement changes trust boundaries and information exposure. Architectural isolation can therefore constrain the impact of injected instructions even when semantic detection fails, although excessive separation may reduce the contextual information required for effective coordination.

Recognizing the limitations of static defenses, Runtime Verification checks whether proposed actions remain authorized and safe during execution. Task Shield [102] verifies tool calls against the user’s original task, while MELON [103] detects indirect injection through behavioral divergence between masked and unmasked re-execution. ShieldAgent [104] translates natural-language policies into an action-level safety model, and IPIGuard [105] constrains execution to legal paths in a tool-dependency graph. DRIFT [106] combines secure planning with dynamic rule isolation, whereas ALRPHFS [107] prioritizes high-confidence risks and escalates ambiguous cases to slower reasoning. More recent mechanisms include executable safety constraints in GuardAgent [108], intent verification before irreversible actions in VIGIL [109], and causal ablation of security-critical decisions in CausalArmor [110]. Zhang et al. [111] propose DUDE, which combines hybrid-reward learning with experience summarization to evaluate deceptive interface interactions while preserving task completion. Trajectory-level causal analysis in CausalTrace [112] and look-ahead simulation in Speculative Safety Honeypot [113] provide additional runtime signals for detecting or avoiding unsafe actions.

Collectively, defenses against external interaction attacks combine early filtering, architectural containment, and action-level verification. These layers are complementary: sanitization reduces the likelihood that malicious control reaches the reasoning core, isolation limits the resources exposed after a bypass, and runtime verification evaluates whether a proposed action remains consistent with user intent and safety policy. Their effectiveness nevertheless depends on model accessibility, policy coverage, verification reliability, computational overhead, and the preservation of benign task utility.

Future Outlook. We anticipate that defenses against external interaction attacks will evolve primarily along two dimensions: contextual precision and deployment efficiency. Rather than classifying an entire external input as either safe or malicious, future mechanisms may track provenance, instructional authority, and downstream influence at the level of individual content segments and proposed actions. In closed-model or latency-sensitive settings, layered defenses could activate expensive verification only when lightweight checks identify a high-risk interaction. Their effectiveness will depend on preserving benign task utility while controlling false rejection, computational overhead, and exposure to adaptive attacks.

6.2. Defenses against Internal Cognitive Attacks

Mitigation strategies for internal cognitive attacks aim to fortify the agent’s reasoning backbone and memory integrity, ensuring the cognitive core remains robust against logic hijacking, memory poisoning, and backdoor triggers. Unlike external defenses that filter inputs at the boundary, these approaches focus on internalizing safety directly into the agent’s planning, retrieval, and parameter optimization processes. To organize the fragmented literature, we classify mitigation strategies into three critical dimensions: Reasoning & Planning, Memory Integrity, and Internal Alignment.

In the domain of Reasoning and Planning, defenses introduce consistency checking, validated execution, risk simulation, and explicit abstention into the cognitive loop. Li et al. [114] propose ReAgent, which checks consistency among thoughts, actions, reconstructed instructions, and the original user query. Feng et al. [115] propose AgentRR, which restricts execution to previously validated experience traces, while Bonagiri et al. [116] train agents to recognize high-risk ambiguity and quit selectively rather than continue unsafe execution. SafeHarbor [117] learns hierarchical memory-supported decision boundaries for agent guardrails, whereas SafeAgent [118] uses an automated risk simulator to generate and refine safety supervision. These mechanisms intervene at different stages of decision formation, from checking an evolving reasoning trace to constraining, simulating, or terminating a proposed plan.

For Memory Integrity, defenses control what enters memory, what is retrieved, and how stored information influences subsequent decisions. Wei et al. [119] introduce A-MemGuard, combining consensus-based validation with a dual-memory structure to identify poisoned entries and learn from prior failures. Mao et al. [120] propose AgentSafe and its HierarCache, which separates data by security level and quarantines malicious streams. Sunil et al. [121] assign composite trust scores and temporal decay to memory entries before retrieval. For multimodal memory, COGNITIVEGUARD [68] screens stored visual information before it influences long-horizon recommendation planning. A detection–reflection procedure evaluated with PS-Bench [122] identifies risky personalized contexts and prompts the agent to reconsider responses that may have been legitimized by benign interaction history. Memory defenses must therefore protect the complete lifecycle of persistent state, including admission, storage, retrieval, interpretation, and downstream use.

The dimension of Internal Alignment aims to internalize robust behavior within agent policies rather than relying exclusively on external checks. Pan et al. [123] propose AdvEvo-MARL, which co-evolves attacker and defender agents through adversarial multi-agent reinforcement learning. Patlan et al. [124] show that instruction fine-tuning can reduce susceptibility to context manipulation under their evaluated conditions. PrivAct [125] trains contextual privacy preferences through multi-agent feedback, whereas Privacy Collapse [126] demonstrates that benign fine-tuning can erode contextual privacy behavior even when conventional safety and utility indicators remain stable. Internal alignment must therefore be treated as a lifecycle property that can be strengthened, preserved, or degraded by subsequent adaptation.

Collectively, defenses against internal cognitive attacks operate at three complementary levels: reasoning controls constrain how decisions are formed, memory controls regulate the persistent state on which later decisions depend, and alignment methods shape the policies that govern agent behavior. Their common objective is to prevent unsafe internal changes from becoming authorized external actions. However, assurance remains difficult because cognitive state is only partially observable, protective reasoning introduces computational overhead, and fine-tuning or long-term adaptation may alter previously validated behavior.

Future Outlook. We anticipate that defenses for internal cognition will evolve primarily along two dimensions: continuous state assurance and lifecycle re-evaluation. Future mechanisms may track how intermediate decisions, retrieved memories, and tool-linked reasoning alter an agent’s trajectory before those changes become externally visible. Because fine-tuning, memory updates, and long-horizon interaction can change internal behavior after deployment, assurance may need to be repeated throughout the agent lifecycle rather than established once. A central challenge will be distinguishing legitimate adaptation from poisoning, backdoor activation, or gradual reasoning manipulation when internal state is only partially observable.

6.3. Defenses against Multi-Agent Collaboration Attacks

Mitigation strategies for multi-agent collaboration attacks aim to secure the collective intelligence against cascading failures, hallucination propagation, and mole agents. Unlike isolated defenses that focus on individual nodes, these approaches operate on the interaction topology and consensus protocols to prevent malicious influence from spreading across the network. To organize the emerging literature, we classify mitigation strategies into three critical dimensions: Topology & Graph Analysis, Behavioral Profiling & Audits, and Consensus & Protection.

In the domain of Topology and Graph Analysis, defenses model communication structure to detect anomalous influence and interrupt propagation. Wang et al. [127] propose G-Safeguard, which applies graph neural networks to multi-agent utterance graphs for anomaly detection and topological intervention. Zhou et al. [128] introduce GUARDIAN, modeling interactions as temporal attributed graphs and pruning anomalous nodes through an information-bottleneck mechanism. Miao et al. [129] propose BlindGuard, which combines hierarchical graph encoding with bidirectional edge pruning without requiring labeled attack data, while Zhou et al. [130] introduce INFA-GUARD to distinguish root attackers from infected agents and apply different remediation strategies to each. XG-Guard [131] uses bi-level graph anomaly detection to identify suspicious agents and interactions while producing graph-level explanations. Architecture Matters [132] further shows that topology, role allocation, and memory visibility affect attack success even without an explicit detector. ResMAS [133] jointly optimizes topology and role prompts for resilience to random agent failures, providing structural evidence while addressing a different failure model from malicious propagation.

Behavioral Profiling and Audits identify compromised participants from their expressed behavior, reasoning consistency, internal activations, or peer evaluations. Xie et al. [85] propose AgentXposed, combining HEXACO-based behavioral profiling with targeted interrogation to expose covert adversaries. Chen et al. [134] introduce PCDC, which uses psychometric screening and a dedicated enforcement agent to identify and isolate risky participants. Fan et al. [135] propose PeerGuard, in which agents cross-check their peers’ reasoning to identify unreliable collaborators, while Huang et al. [84] employ Challenger and Inspector roles to question and audit peer outputs. The Doctor and Police defenses in PsySafe [87] respectively repair risky agent profiles and introduce supervisory critique. AcMAS [136] detects malicious participants from activation patterns, providing a model-internal signal that complements observable behavior and peer assessment.

In the dimension of Consensus and Protection, defenses constrain how collective decisions are formed and how influence flows through critical roles. Hu et al. [137] apply randomized smoothing to collective decisions, providing probabilistic robustness under bounded perturbations. CP-WBFT [138] adapts Byzantine fault-tolerant consensus to heterogeneous agent reliability. Wen et al. [139] propose CoTGuard, embedding task-specific triggers in reasoning chains to detect unauthorized reproduction. Adv-IMBIA [89] protects critical roles and workflow stages against role-based compromise in multi-agent software-development teams, while ControlValve [80] constrains permissible control-flow paths among collaborating agents. These mechanisms address different protection objectives, ranging from decision robustness and intellectual-property tracing to workflow containment.

Collectively, defenses against multi-agent collaboration attacks operate across system structure, participant behavior, and collective decision protocols. Topology-based mechanisms limit the paths through which malicious influence can spread, behavioral auditing identifies compromised or unreliable participants, and consensus or workflow controls protect shared outcomes after individual components become untrustworthy. Their effectiveness nevertheless depends on the availability of reliable structural and behavioral evidence, as well as the communication and computational overhead introduced by collaborative supervision.

Future Outlook. We anticipate that defenses for multi-agent collaboration will evolve primarily along two dimensions: topology awareness and adaptive trust. Instead of assigning fixed reputation or applying uniform consensus rules, future systems may update trust according to observed behavior, role criticality, communication paths, and the downstream influence of each participant. These signals could be combined with protocol validation, graph-based anomaly analysis, and Byzantine-resilient decision mechanisms to isolate compromised influence while preserving useful coordination. The resulting guarantees will remain conditional on explicit assumptions about collusion, participant independence, topology visibility, and recoverability after partial compromise.

7. Security Frameworks and Evaluation Benchmarks

7.1. Security Frameworks

Existing security frameworks can be organized into three complementary paradigms: System-Level Isolation and Architecture Design, Policy Enforcement and Behavioral Alignment, and Runtime Supervision and Collaborative Monitoring. These paradigms respectively establish trust boundaries, translate policies into behavioral constraints, and supervise agent behavior during operation.

7.1.1. System-Level Isolation and Architecture Design

To mitigate systemic risks from direct resource access, a trusted computing base and explicit isolation boundaries are primary defenses. AIOS [140] uses an OS-inspired kernel and Access Manager. ISOLATEGPT [98] assigns isolated environments to applications and routes communication through a trusted hub, while AirGapAgent [99] minimizes contextual disclosure. Other designs combine sandboxing with protected computation [141]. ACE [142] derives a trusted abstract plan from the user query, concretizes it with untrusted application content, statically checks information flow, and executes components under data and capability isolation. Its restricted runtime is not claimed to provide a general formal soundness guarantee.

Architecture can also be analyzed as a security variable rather than a passive container. MaMa [143] formulates agentic-system design as a Stackelberg security game, while Architecture Matters [132] measures how role allocation, topology, and memory visibility change attack success. EcoAgent [101] illustrates device–cloud partitioning and on-device verification but evaluates efficiency and functional privacy rather than adversarial resistance. Privacy-R1 [144] provides a narrower example by routing sub-prompts between a local and a remote model in a single-turn setting.

7.1.2. Policy Enforcement and Behavioral Alignment

Building on architectural isolation, secure agents require explicit mechanisms that translate user intent, constitutions, and regulatory requirements into enforceable behavioral constraints. Hua et al. [145] propose TrustAgent, combining pre-planning safety learning, planning-time constitutional guidance, and post-planning inspection. Chen et al. [104] introduce ShieldAgent, converting unstructured policies into an Action-based Safety Policy Model for action-level verification. Zhang et al. [146] discuss a neuro-symbolic direction that translates requirements into formal constraints for plan verification. At the multi-agent level, AGENTBREEDER [147] searches collaboration structures through evolutionary optimization, Shapley-Coop [148] uses Shapley-value incentives to address free-riding, and defense-in-depth governance [149] distributes oversight across the collaboration lifecycle.

SAGA [150] provides provider-mediated registration and discovery, secure inter-agent communication, and encrypted access-control tokens with time- and request-bounded user contact policies. Data Permissions [151] predicts users’ context-dependent sharing preferences to reduce repeated permission prompts; reliable enforcement, revocation, and policy compliance remain open problems rather than implemented guarantees. Task Shield [102] enforces the narrower invariant that each tool call remain aligned with the user’s original task, while Privacy-R1 [144] studies single-turn privacy-aware routing between a local and a remote model.

7.1.3. Runtime Supervision and Collaborative Monitoring

Because static policies and architectural boundaries cannot anticipate every operational failure, agent systems also require mechanisms for execution monitoring, monitor evaluation, and incident response. Sentinel Agents [152] distribute semantic-anomaly and collusion monitoring across a shared conversational space and coordinate isolation through a central controller.LlamaFirewall [153] combines prompt-injection detection, reasoning-trace alignment checks, and code-oriented analysis in a system-level guardrail.

AGrail [154] generates task-specific safety checks and updates safety memory through test-time optimization, supporting adaptive pre-action guardrails. Weak-to-Strong Monitoring [155] instead red-teams monitors on completed trajectories and studies whether weaker models can detect covert behavior by stronger agents; it evaluates monitor reliability rather than an online runtime-control mechanism. AIR [156] detects, contains, recovers from, and eradicates unsafe states before synthesizing rules intended to prevent recurrence.

Taken together, these frameworks distribute security responsibilities across architecture, policy, and system operation. Architectural mechanisms establish trust boundaries and limit resource exposure, policy frameworks translate user intent and regulatory requirements into behavioral constraints, and runtime mechanisms monitor execution and respond to emerging failures. Effective agent security therefore depends on coordination across these layers rather than on any single framework providing complete protection.

7.2. Security Benchmarks

As LLM agents evolve toward autonomous tool use and environmental interaction, static and single-turn benchmarks alone are insufficient to capture risks arising over complete execution trajectories. Table 8 compares representative benchmarks across five dimensions: domain, interaction mode, threat focus, scale, and key metrics. We organize them into four practical families: Comprehensive Safety and Misuse Evaluation, Tool-Centric Interaction Benchmarks, Adversarial Robustness and Prompt Injection, and Domain-Specific Risk Assessment. These families reflect primary evaluation objectives and are not mutually exclusive; the table therefore presents a curated set of formally published and mechanistically distinct resources rather than an exhaustive inventory. Because scale is reported in benchmark-specific units, values are not directly comparable across rows.

7.2.1. Comprehensive Safety and Misuse Evaluation

AgentHarm [157] evaluates malicious capabilities over 110 multi-step tasks, whereas OpenAgentSafety [171] evaluates safety constraints in realistic long-horizon tool use. ASSEBench [172] studies subtle multi-step safety failures with a memory-augmented auditing setup, and ALI-Agent [173] generates long-tail alignment cases through staged emulation and refinement. Among newer suites, PropensityBench [158] studies harmful-action propensity under pressure using 5,874 scenarios and 6,648 tools; MT-AgentRisk [159] evaluates distributed harmful intent across 365 multi-turn tasks and five tools; and TAMAS [160] covers multi-agent adversarial risk with 300 adversarial and 100 benign tasks across five domains, six attack types, and 211 tools.

7.2.2. Tool-Centric Interaction Benchmarks

ToolEmu [174] simulates tool environments and underspecified instructions, while AgentDojo [161] provides reproducible indirect-injection evaluation in office-tool tasks. ToolFuzz [175] fuzzes tool definitions to expose documentation and runtime defects, and Attractive Metadata Attack evaluation [52] measures tool-selection manipulation through optimized descriptions. ToolSword [162] covers six safety scenarios across input, execution, and output stages and compares eleven models. ASB [176] evaluates mixed threats across more than 400 tools. For standardized tool ecosystems, MSB [177] contains 12 MCP-specific attacks, 65 tasks, and 709 tools; MCP-SafetyBench [163] contains 245 cases across five domains and 20 attack classes; and MCPTox [55] evaluates malicious instructions in tool descriptions on 45 live servers, 353 tools, and 1,348 cases.

7.2.3. Adversarial Robustness and Prompt Injection

WASP [164] evaluates end-to-end web attacks and exposes “security by incompetence.” The Agent Robustness Evaluation framework [178] models an agent as a computation graph to study how adversarial effects propagate, while ShieldAgent-Bench [104] evaluates guardrail compliance over adversarial trajectories. SafeSearch [38] contains 300 sandboxed search cases across five risk types, and AgentLAB [179] evaluates adaptive long-horizon attacks in 644 cases and 28 environments. ART [165] analyzes a public competition containing approximately 1.8 million submissions against 22 agents in 44 deployment scenarios. VPI-Bench [42] contains 306 interactive visual-injection cases on five computer-use platforms. The b³ benchmark [180] draws from a pool of 194,331 submissions but uses 210 selected attacks for the formal evaluation of 34 backbone LLMs. ACIArena [166] evaluates cascading injection across six multi-agent implementations with 1,356 cases.

7.2.4. Domain-Specific Risk Assessment

PrivacyLens [167] tests whether privacy-norm knowledge transfers from probing questions to executable trajectories using 493 privacy-sensitive seeds, vignettes, and trajectories. BrowserART [168] shows that chat refusal does not reliably transfer to browser-agent action across 100 harmful behaviors and 40 synthetic websites. SAPA-Bench [169] evaluates privacy awareness in 7,138 smartphone-agent scenarios. ST-WebAgentBench [170] contains 375 policy-constrained web tasks carrying 3,057 policy instances across six safety-and-trustworthiness dimensions. Earlier domain-specific resources include RedCode [181] and CodeBreaker [182] for unsafe code-agent behavior, CVE-Bench [183] for vulnerability-exploitation capability, and SafeArena [184] for misuse compliance in web interaction. Additional resources include RedCodeAgent [185], AgentDoS [186], AgentDAM [187], GPT Spills [62], unsafe-patch evaluation [188], and deceptive-interface benchmarks [48,111].

Taken together, existing benchmarks differ substantially in threat assumptions, interaction length, environmental realism, and evaluation objectives. General-purpose suites provide broad coverage, whereas tool-centric, adversarial, and domain-specific benchmarks offer greater depth for particular attack surfaces. Their results should therefore be interpreted together with task utility, execution conditions, scale definitions, and reproducibility constraints rather than compared through raw scores alone.

8. Future Work

Architectural Evolution towards Intrinsic Security: 

Existing evidence shows that role allocation, topology, and memory visibility materially affect attack success [132], while adversarial architecture search can optimize configurations against compromised participants [143]. These findings motivate architectures that enforce least privilege across tools, memory, credentials, network access, and persistent resources while making their trusted components and permitted information flows explicit. Resource governance should be part of the same design: AgentDoS demonstrates that autonomy without lifecycle controls can create exploitable availability failures [186]. A central research question is how to combine access control, isolation, and per-task or per-session quotas without undermining useful autonomy.

Standardization of Agent Identity and Trust Protocols: 

MCP evaluations expose protocol-level trust failures involving metadata, server identity, host policy, tool composition, and session state [163,177]. These failures motivate authenticated agent and tool identities, scoped credential delegation, consent provenance, capability manifests, revocation, and auditable sessions. Signatures or decentralized identifiers may support these controls, but they do not by themselves establish reputation, authorization, or trustworthy behavior. Protocol proposals should therefore specify which entity authenticates each claim, how authority is delegated and revoked, and how compromised participants or stale metadata are contained.

Integration of Confidential Computing and Verifiable Reasoning: 

High-stakes deployments may benefit from confidential-computing mechanisms that protect memory, credentials, and tool state, but their trusted computing base, side channels, rollback behavior, and interaction with external tools require explicit evaluation. Verifiable reasoning remains a more speculative agenda: work must first define machine-checkable statements about plans, tool calls, data access, or policy constraints. Zero-knowledge proofs may attest to such statements without revealing protected inputs, but cannot by themselves prove that unrestricted natural-language reasoning is semantically safe.

Theoretical Foundations based on Game Theory and Formal Methods: 

MaMa provides an initial Stackelberg-game formulation of agentic-system design [143], and CP-WBFT studies reliability under Byzantine participants [138]; both still depend on explicit empirical settings and attacker assumptions. Future work should define capabilities, utilities, equilibrium assumptions, fault bounds, and robustness under task or model shift. Formal methods should target explicit artifacts such as capability policies, resource invariants, communication graphs, tool-call preconditions, and executable plans, providing narrow but defensible guarantees rather than claiming provable security for unrestricted cognition.

Socio-Technical Alignment and Liability Mechanisms: 

Operational governance must include prevention, response, and accountable recovery. AIR shows how detection, containment, remediation, and recurrence-prevention rules can be incorporated into an agent loop [156], while CausalTrace illustrates counterfactual attribution over execution trajectories [112]. Future audit standards should record authenticated requests, retrieved evidence, policy decisions, tool invocations, incident actions, and relevant state transitions while protecting user privacy. Storing Plan-of-Thought or Chain-of-Thought snapshots does not automatically establish causality or liability; evidentiary standards, logging boundaries, failure conditions, and trusted components must be specified.

9. Conclusions

In this survey, we have presented a comprehensive systematization of the security landscape for LLM-based agents, identifying how vulnerabilities are reshaped and amplified when large language models transition from passive text generators to autonomous decision-makers. Unlike traditional adversarial attacks on standalone models, our analysis reveals that the security risks of agents are deeply rooted in their structural autonomy, specifically within the interfaces of environment perception, the cognitive loops of reasoning and memory, and the trusted channels of multi-agent collaboration.

By establishing a novel taxonomy centered on the structural dimensions of External Interaction, Internal Cognition, and Multi-Agent Collaboration, we have elucidated how adversaries can exploit the conflation of instruction and data, the persistence of memory states, and the implicit trust in coordination protocols to subvert agent behaviors. Furthermore, we have mapped the corresponding defense landscape, highlighting the critical transition from static input filtering to dynamic, system-level resilience mechanisms.

As LLM agents are increasingly integrated into critical infrastructure and open-ended environments, security can no longer be treated as an afterthought. It must be designed into agentic architectures through explicit trust boundaries, authenticated and least-privilege tool access, protected memory and state, runtime verification, resilient collaboration, incident response, and evaluation under realistic long-horizon adversaries. We hope this survey serves as a roadmap for researchers and practitioners, fostering the development of next-generation agents that are not only capable and autonomous but also more secure, trustworthy, and rigorously evaluated.