Context Compression for Long-Horizon AI Agents: Lifecycle Management, Token Economics, and Production Patterns

1. Introduction

1.1. Why Token Consumption Matters for Agents

A conversational LLM application typically issues a small number of calls per user turn. An autonomous agent, by contrast, executes a loop: it receives a goal, emits a tool call, observes the (often large) result, appends it to the running transcript, and re-submits the entire transcript to the model—repeating until a terminal response is produced. Anthropic reports that single-agent workflows “use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats” Hadfield et al. (2025). Production agents can span hundreds of conversational turns, so context length—and therefore cost and latency—grows monotonically across a session.

Four forces make this growth painful:

1.

Cost. Because the full transcript is re-sent each turn, cumulative input spend can grow quadratically in the number of turns absent caching (a sum of monotonically growing per-turn contexts; Figure 1b); prompt caching softens the constant but not the trend. Unaudited community telemetry for OpenClaw attributes an estimated 40–50% of typical token usage to context accumulation alone, and a single user reported that session context occupied 56–58% of a 400K-token window—roughly 230,000 tokens re-sent with every message LaoZhang AI (2026).

2.

Latency. Prefill compute scales with context length; OpenHands measured average response latency rising to 16 s at iteration 100 without condensation Smith (2025).

3.

Finite context windows. Even with 200K–1M-token windows, long-horizon tasks exceed the window, forcing some form of eviction or summarization Anthropic (2025c).

4.

Accuracy degradation (“context rot”). Performance falls as inputs grow. Liu et al. (2024a) show a U-shaped accuracy curve in which performance “degrades significantly” when relevant information sits in the middle of long inputs (multi-document QA fell from ∼75% when the gold document was first to ∼55% when it was tenth of twenty, across GPT-3.5-Turbo, GPT-4, Claude-1.3, MPT-30B, and LongChat-13B). Chroma’s 2025 Context Rot report extends this to 18 frontier models (including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3), finding that every model degrades as input length grows, even at 50K tokens within a 1M-token window Hong et al. (2025). Controlled probes sharpen the diagnosis: on NoLiMa, which removes literal needle–question lexical overlap, 11 of 13 models advertising ≥128K context fall below half their short-context accuracy at just 32K tokens (GPT-4o drops from 99.3% to 69.7%) Modarressi et al. (2025); RULER finds only about half of 17 evaluated long-context LMs sustain acceptable accuracy at 32K despite claiming 32K+ windows Hsieh et al. (2024); and BABILong reports that models effectively use only 10–20% of their context on facts scattered across long inputs Kuratov et al. (2024). The effective window is thus far smaller than the advertised one—keeping context small often preserves more usable signal than filling it, though the degradation is highly task- and model-dependent (some models and tasks stay near-flat), a heterogeneity we revisit when discussing evaluation in Section 7.

For persistent general-purpose agents the problem is acute because they run even when idle. OpenClaw’s “heartbeat” fires scheduled background turns; a 5-minute heartbeat carrying 50K tokens of context consumes roughly 600,000 input tokens per hour—about $3/hour on a frontier model purely to “stay awake.” This per-hour figure is an arithmetic illustration rather than a measurement; a single misconfigured email-check loop has separately been reported to burn $50 in one day Wu (2026); Hayes (2026). Figure 2 summarizes both sides of this economics: the token multipliers that make agents expensive relative to a chat turn, and the individual levers (caching, tool-surface offloading, condensation) that claw the cost back.

1.2. Scope and Named Systems

This survey foregrounds context compression: methods that reduce the token footprint of the information the model must attend to, while preserving task-relevant content. We use as recurring case studies a cohort of general-purpose agents—domain-agnostic, tool-using, and long-running—rather than narrow task pipelines, and throughout we return to OpenClaw, Hermes, Manus, Claude Code, and OpenHands as the recurring named systems, with MemGPT/Letta and the earlier AutoGPT/BabyAGI lineage as comparison points OpenClaw (2026a); Nous Research (2026a). We defer the detailed per-system mechanisms—bootstrap layouts, condensers, dual context engines, and filesystem offloading—to Section 5, where they are compared side by side. Because our sources span very different levels of scrutiny, we tier evidence explicitly and hold to that tiering throughout: (i) peer-reviewed papers and arXiv preprints are treated as primary evidence; (ii) first-party vendor engineering blogs and product documentation are documented but non-archival, reporting real deployments under settings we cannot independently verify; and (iii) community write-ups, third-party telemetry, and forum reports are unaudited signals, flagged inline at every use and never the sole basis for the survey’s conclusions or recommendations. Cited URLs were checked against their live primary sources at the time of writing (June 2026), and non-archival sources carry access dates in the references. We further caution that the 2025–2026 frontier synthesis (Section 6) rests largely on preprints not yet peer-reviewed, so those particular conclusions may shift as the record matures.

Figure 3 previews the terrain the survey covers: it places the great majority of the methods discussed in the sections that follow at their first public appearance, organized into the four method families developed in Section 3 and Section 4. The plotted set is representative rather than an exhaustive census: a handful of frameworks, vendor engineering features, and methods named only as lead-ins are omitted for legibility. Two trends visible in the figure shape our emphasis throughout: the field’s marked acceleration over 2024–2025, and the 2025–2026 surge of learned, agent-native (RL-trained) compression—the dense bottom lane—whose synthesis (Section 6) is a central contribution of this survey.

Out of scope. We focus on reducing the token footprint of the textual and representational context a tool-using agent repeatedly re-prefills, and deliberately set aside four adjacent problems. (i) Multimodal context—compressing vision or audio tokens for computer-use and GUI agents—is an important but distinct token sink we do not treat. (ii) Output-side reasoning compression—shortening chain-of-thought or tool-call traces as they are generated—is complementary to the input-side compaction studied here. (iii) Alternative long-context architectures—linear-attention and state-space models that lower the per-token cost of long inputs rather than shortening them—substitute for, rather than instantiate, compression. (iv) Information-theoretic limits of lossy context compression remain largely uncharted; our treatment is empirical throughout. We note these to delimit the survey, not to diminish them.

Relation to existing surveys. Prior surveys cover adjacent but narrower scopes. Li et al. (2025b) survey prompt compression (hard and soft methods) for LLMs; the KV-cache management survey of Li et al. (2025a) organizes that single family into token-, model-, and system-level optimizations; the agent-memory survey of Zhang et al. (2024d) catalogs memory mechanisms; Mei et al. (2025) cast compression as one component of a broad “context engineering” pipeline (acquisition, processing, management); and the contemporary memory survey of Du (2026) organizes agent memory as a write–manage–read loop across five mechanism families. None is simultaneously agent-centric, cross-family, and token-efficiency-centric. Our contribution is a lifecycle taxonomy—organized around admission, placement, compaction, recovery, reuse, and governance—into which the four method families nest, oriented to the operating constraints of long-running, tool-using agents (repeated re-prefill, prompt-cache economics, irreversibility, and provenance); and we give particular weight to the 2025–2026 shift toward learned, agent-native compression (Section 6).

Contributions. This survey makes five contributions:

1.

A unified agent-context lifecycle model—admission, placement, compaction, recovery, reuse, and governance—that frames context compression as full-session management rather than one-shot prompt shortening (Section 3); governance is included as a normative stage—motivated by compression–security interactions but not yet instantiated in any surveyed production system.

2.

A taxonomy by intervention point and reversibility that nests the four method families (hard/natural-language, soft/embedding, KV-cache, and complementary) inside the lifecycle, rather than treating them as parallel silos (Section 3 and Section 4).

3.

A production-agent comparison of how deployed general-purpose agents (OpenClaw, Hermes, Manus, Claude Code, OpenHands) instantiate these choices in practice (Section 5).

4.

A synthesis of the 2025–2026 shift from compression-as-scaffold to compression-as-policy, where the compaction step becomes a reinforcement-learned action (Section 6).

5.

An evaluation agenda that jointly measures task success, peak and total tokens, latency, cache hit rate, and recovery/provenance, rather than compression ratio alone (Section 7).

Table 1. How this survey differs from adjacent survey families. Prior surveys each isolate one framing of the problem; this survey targets full-session token economics and the compression decisions a tool-using agent must make over an entire run.

Survey family	Primary focus
Prompt compression Li et al. (2025b)	Prompt representation and hard/soft compression mechanisms for a single call.
KV-cache / serving Li et al. (2025a)	Serving-time memory, throughput, eviction, and quantization of the attention cache.
Agent memory Zhang et al. (2024d); Du (2026)	Persistent memory mechanisms as a write–manage–read loop.
Context engineering Mei et al. (2025)	The broad acquisition–processing–management pipeline, with compression as one component.
This survey	Full-session token economics and compression decisions in tool-using agents.

Table 1. How this survey differs from adjacent survey families. Prior surveys each isolate one framing of the problem; this survey targets full-session token economics and the compression decisions a tool-using agent must make over an entire run.

Survey family	Primary focus
Prompt compression Li et al. (2025b)	Prompt representation and hard/soft compression mechanisms for a single call.
KV-cache / serving Li et al. (2025a)	Serving-time memory, throughput, eviction, and quantization of the attention cache.
Agent memory Zhang et al. (2024d); Du (2026)	Persistent memory mechanisms as a write–manage–read loop.
Context engineering Mei et al. (2025)	The broad acquisition–processing–management pipeline, with compression as one component.
This survey	Full-session token economics and compression decisions in tool-using agents.

2. What Makes Agent Compression Different

Compressing an agent’s context is not the same problem as compressing a long prompt. An agent does not read its input once; it accumulates a transcript over many turns, re-feeds that transcript to the model on every step, and pays for tools, schemas, and provenance that a one-shot prompt never carries. Five properties separate the agent setting from the static long-context setting these methods were originally designed for, and they motivate the lifecycle view we adopt in Section 3.

Repeated re-prefill and quadratic session cost. An agent transcript is not a long prompt that the model consumes once: the full history is re-sent to the model on every turn. Because each per-turn context is itself monotonically growing, cumulative input spend can grow quadratically in the number of turns absent caching—a sum of per-turn contexts, not a single pass over a fixed input. Prompt caching softens the constant but not the trend. The effect is large in production—unaudited community telemetry attributes an estimated 40–50% of typical token usage to context accumulation alone (the OpenClaw figures of Section 1). This is why context that merely exists is a recurring cost, and why multi-agent architectures that fan work out across isolated sub-agents can burn far more tokens than a single conversation Hadfield et al. (2025). A method that shaves a fixed number of tokens once is worth little; what matters is bending the per-turn growth curve.

Tool observations and schemas as hidden token sinks. A static prompt is a single block of text; an agent context is a heterogeneous mix of messages, tool definitions, tool outputs, scratchpads, memory files, and sub-agent summaries, and these components have very different cost profiles. Tool observations—the environment outputs returned after each action—dominate: JetBrains’ Complexity Trap study finds observations make up roughly 84% of an average SWE-agent turn, so the bulk of compressible mass lives there rather than in the model’s own reasoning Lindenbauer et al. (2025). This 84% is a single-scaffold, single-study measurement—indicative of where the mass sits rather than a universal constant. Tool definitions and inter-tool data are a separate, often overlooked sink: exposing MCP tool servers as code APIs reduced a representative workflow from 150,000 to 2,000 tokens Anthropic (2025b), and on-demand tool discovery cut tool-definition overhead by ∼85% Anthropic (2025a) (Section 4.4 details both). A compression scheme that treats the context as undifferentiated text misses that these sinks have distinct origins and distinct, often cheaper, remedies.

Prompt caching changes the optimization target. In the static setting, fewer tokens is unambiguously cheaper. For agents that reuse a stable prefix across turns, the optimization target is no longer raw length: a shorter prompt can be more expensive if it destroys a cached prefix. The economics are concrete. Anthropic cache writes cost 1.25× (5-min TTL) or 2.0× (1-hour TTL) base input while cache reads cost only 0.1×—a 90% discount, break-even at ∼2 hits Anthropic (2024); OpenAI applies an automatic 50% discount on cached prefixes over 1,024 tokens OpenAI (2024). A compaction event that rewrites the cached prefix forfeits these savings, so naive condensation can be a net cost increase. Production agents lean heavily on this discount—Hermes reports prompt caching reduces input token costs by ∼75% on multi-turn conversations Nous Research (2026a), and OpenClaw auto-enables caching for API-key sessions centminmod (2026)—which means any compression operator must be co-designed with the cache boundary rather than evaluated on token count alone.

Irreversibility and recoverability. Summarization is not interchangeable with the alternatives that preserve a path back to the original. Lossy compaction discards tokens whose future relevance is unknown, and the loss can be severe: one analysis observed autocompaction reduce 132K tokens to 2.3K (98% reduction), discarding a session’s hard-won understanding Santoni (2026). Three recoverable designs avoid this. Filesystem offloading keeps a pointer rather than the content—Manus drops a web page’s text as long as the URL is preserved and omits a document’s body if its path remains, so context shrinks “without permanent loss” Ji (2025), and Anthropic’s Managed Agents generalize this to a durable session log living outside the model’s window Anthropic (2026a). View-level condensation operates by inserting markers and never mutating the raw event stream, so sessions remain replayable All Hands AI (2025a). Inside the KV cache, the analogous move is reconstruction-scored compression: KVzip scores each KV pair by how much it supports reconstructing the original context, yielding a reusable compressed cache rather than query-conditioned eviction Kim et al. (2025). Whether an operator is lossy or recoverable is therefore a first-class property in the agent setting, not an implementation detail.

Provenance and privilege boundaries. A static prompt carries one trust level; an agent context mixes system instructions, user input, tool outputs, and untrusted environment content in a unified context whose tokens influence generation regardless of provenance. Compression can blur these boundaries with safety consequences: a study of compression methods finds system-prompt leakage rises with compression, because defense and guardrail instructions often sit in the (evictable) system prompt and aggressive KV compression can silently disable them Chen et al. (2025). The flat context model is itself exploitable: on an isolated testbed running unmodified OpenClaw, ClawWorm shows that a single-message prompt-injection chain can achieve persistent, self-propagating compromise by combining flat context trust with unconditional loading of agent configuration Zhang et al. (2026). Background heartbeat turns provide a separate unattended path through which persistent state can be read or updated OpenClaw (2026b). A compression scheme that ignores which part of the context a token came from can therefore degrade not just accuracy but security.

These five properties—quadratic re-prefill cost, heterogeneous token sinks, the caching-aware optimization target, the lossy/recoverable distinction, and provenance boundaries—are what separate this survey from prompt-compression and KV-cache surveys, which treat the context as a single static block of fixed provenance read exactly once.

3. A Lifecycle Taxonomy for Agent Context Compression

We formalize the agent context-management problem and then partition methods.

Problem setup. Let an agent’s interaction history at step t be a sequence of events $H_t=(e_1,e_2,\dots ,e_t)$, where each $e_i$ is a message, tool call, or observation with token length $|e_i|$. The model input at step t is a context $C_t=\mathsf{\Phi }\left(H_t\right)$ assembled by a context function $\mathsf{\Phi }$. A naive agent sets $\mathsf{\Phi }\left(H_t\right)=\mathrm{concat}\left(H_t\right)$, so $|C_t|={\sum }_{i\le t}\left|e_i\right|$ grows without bound. A compression operator $\kappa$ produces ${\tilde{C}}_t=\kappa \left(C_t\right)$ with $|{\tilde{C}}_t|\ll |C_t|$. We define the compression ratio $\rho =|C_t|/|{\tilde{C}}_t|$ and the retention $R=\mathrm{Perf}\left({\tilde{C}}_t\right)/\mathrm{Perf}\left(C_t\right)$, the fraction of downstream task performance preserved. The central design tension is to maximize $\rho$ subject to $R\approx 1$.

The agent-context lifecycle. Rather than partitioning methods first by mechanism, we organize them by when in an agent’s context-management lifecycle they intervene. Every event $e_i$ that enters $H_t$ passes through a sequence of decisions: whether it is admitted into context at all, where it is placed, how stale context is compacted, whether discarded detail can be recovered, whether the prefix can be reused across calls, and how the resulting context is governed for provenance and security. Table 2 states the question each stage answers and the representative mechanisms that operate there. This lifecycle view and the four-family where-axis introduced below are complementary but correlated lenses on the same methods rather than independent axes: a single system typically acts at several stages and the families slot into the lifecycle rather than competing with it, but the stages are far from uniformly populated across families. Recovery concentrates in the complementary family—filesystem pointers, retrieval stores, and raw transcripts—with reconstruction-scored KV compression (KVzip Kim et al. (2025)) the notable within-model exception; reuse is dominated by provider-side prompt caching; and governance is, so far, a largely normative stage—motivated by compression–security findings Chen et al. (2025); Zhang et al. (2026) and emerging research such as provenance-preserving delta updates (ACE Zhang et al. (2025a)), but not yet a documented practice in any surveyed production agent (Section 5.5).

Mapping the four families onto the lifecycle. The four method families that follow each occupy a characteristic region of this lifecycle. Token-space (hard/natural-language) methods mainly perform admission and compaction, editing the text before it reaches the model. Soft-token (embedding) methods perform compaction inside learned representations, replacing spans with continuous vectors. KV-cache methods target serving-time placement and compaction of the model’s internal key–value memory. Offloading, memory, RAG, and sub-agent strategies target placement and recovery, relocating content outside the window and re-fetching it on demand. The emerging learned, agent-native line cuts across these: it decides compaction and placement as actions taken by the agent’s own policy (Section 6) rather than by an external scaffold; Figure 4 consolidates the lifecycle and marks which families act at each stage.

One family axis and three cross-cutting axes. The primary axis—where compression acts—defines the four families: token space (natural-language text the model reads), embedding space (continuous soft tokens), the KV cache (the model’s internal key–value memory), or outside the context window (complementary strategies). Three further axes cut across all four:

• Lossy vs. recoverable. Lossy operators (summarization, eviction) discard information irreversibly; recoverable operators (filesystem offloading, externalized context objects, reconstruction-scored caches) keep a pointer so the full content can be re-fetched Ji (2025); Anthropic (2026a).
• Training-free vs. model-modifying. Token-space methods leave the model untouched and run on any closed API; embedding- and KV-space methods require model access or self-hosting. This axis directly shapes the recommendations of Section 8.
• External scaffold vs. learned policy. Classically compression is a wrapper around an unmodified agent; an emerging line instead trains the agent to compress its own context (Section 6). Learned-policy compression cuts across families and is the field’s most active frontier.

The soft/embedding and KV boundaries form a continuum rather than a hard partition—learned soft tokens are frequently materialized as cached KV values (e.g. Activation Beacon Zhang et al. (2024a), 500xCompressor Li et al. (2025c))—so we assign convergent methods by their primary mechanism.

This yields four families, detailed in Section 4: (1) hard/natural-language compression; (2) soft/embedding compression; (3) KV-cache compression; (4) complementary strategies (offloading, memory hierarchies, RAG, sub-agent isolation); Figure 3 (Section 1) traces their evolution from 2023 to 2026 and the recent shift toward recoverable and learned, agent-native compression.

4. Method Families as Implementation Choices

With the agent-centric setup in place, we now survey the method families—but rather than catalog them in isolation, we examine each by the same agent-design questions: where it intervenes in the lifecycle (Section 3); whether it is compatible with closed, API-only models or instead requires modifying the model; whether it preserves cacheable prefixes or invalidates the KV-cache on every edit; whether the compression is recoverable or destructive; whether it preserves provenance (a traceable link back to the source span); and which agent workloads it benefits or breaks. Read this way, the families are not competing schools but a menu of implementation choices, each occupying a different point on the lossy↔recoverable, training-free↔model-modifying, and external-scaffold↔learned-policy axes. Figure 5 locates the three in-context points where compression can intervene, and Figure 6 maps the four families against the two cross-cutting axes—together the what/how complement to the lifecycle’s when (Figure 4).

4.1. Natural-Language Compaction

Natural-language compaction keeps the model and API unchanged—it edits the text sent to the model—so it is training-free and closed-API-compatible, and it acts at the admission and compaction stages of the lifecycle. It splits into abstractive (generate a shorter paraphrase/summary) and extractive/pruning (select or delete spans) approaches Li et al. (2025b); Li et al. (2023).

4.1.1. Summarization-Based Compression

Recursive and hierarchical summarization (the conceptual basis). The idea of recursively folding history into a bounded, hierarchical summary—Wang et al. (2023) for long dialogue, Generative Agents Park et al. (2023) for low-level activity into reflections—is the conceptual basis for the “compaction” used in nearly every production agent, and (as Section 6 shows) for the RL-trained compaction policies of 2025–2026.

Trajectory / conversation compaction. OpenHands implements an LLMSummarizingCondenser (a RollingCondenser) that, once history exceeds a size limit, summarizes older events while keeping recent messages verbatim, encoding the user’s goals, progress made, and remaining work plus technical artifacts (critical files, failing tests) All Hands AI (2025b). Anthropic frames “the art of compaction” as choosing what to keep versus discard, recommending engineers “maximize recall” first then tune precision Anthropic (2025c). Claude Code triggers automatic server-side compaction at roughly 80% of the context window (∼160K of 200K tokens), and Claude Opus 4.6 exposes context compaction triggered at a configurable threshold (e.g. 50K tokens) to extend agentic runs Anthropic (2026b).

Observation masking—a cheaper alternative. A counterintuitive result cautions against assuming summarization is always best. JetBrains’ Complexity Trap study finds that simple observation masking (replacing old environment observations with placeholders while keeping reasoning and actions intact) matches or beats LLM summarization on SWE-bench Verified: Qwen3-Coder-480B scored 2.6% higher solve rate at 52% lower cost, and summarization actually inflated trajectory length by 13–15% by obscuring natural stopping signals. Because observations dominate the turn (the ∼84% of Section 2), masking captures most of the available savings (a hybrid masking-plus-summary scheme adds a further 7–11% reduction); the masking-window hyperparameter must, however, be tuned per scaffold, since agents that retain failed retries (OpenHands) need larger windows than those that skip them (SWE-agent) Lindenbauer et al. (2025).

Optimizing the compressor (ACON). ACON optimizes the compression guideline itself via natural-language failure analysis, then distills the compressor into a smaller model. It reports 26–54% reduction in peak token usage on AppWorld, OfficeBench, and Multi-objective QA, is gradient-free, retains over 95% accuracy once distilled, and even improves smaller-LM agents by up to 46% relative to a naive prompting baseline Kang et al. (2025b).

Agent-controlled active compression (Focus). Focus lets the agent itself decide when to consolidate learnings into a persistent “Knowledge” block and prune raw history. On five context-intensive SWE-bench instances with Claude Haiku 4.5, aggressive prompting (compress every 10–15 tool calls) achieved a 22.7% token reduction (14.9M → 11.5M) at identical accuracy (3/5), with per-instance savings of 18–57%; passive prompting yielded only ∼6% savings and degraded accuracy, showing that compression scaffolding must be aggressive to pay off Verma (2026).

4.1.2. Token Pruning/Selective Context Dropping

Pruning deletes low-information tokens before sending the prompt. Selective-Context Li et al. (2023) scores tokens by self-information from a small LM and drops the lowest. The LLMLingua family Jiang et al. (2023b); Microsoft Research (2024) is the canonical line:

• LLMLingua Jiang et al. (2023a): a coarse-to-fine pipeline (budget controller + iterative token-level perplexity pruning + distribution alignment) using a small LM (e.g. GPT-2/LLaMA-7B). It reports “up to 20x compression with only a 1.5 point performance drop” across GSM8K, BBH, ShareGPT, and Arxiv-March23 (LLaMA-7B compressor, GPT-3.5-Turbo target).
• LongLLMLingua Jiang et al. (2024b): adds query-aware coarse-to-fine compression and document reordering for long-context RAG, boosting performance by up to 21.4% with ∼4× fewer tokens on NaturalQuestions, achieving a 94.0% cost reduction on LooGLE, and accelerating end-to-end latency 1.4–2.6× at 2–6× compression of ∼10K-token prompts.
• LLMLingua-2 Pan et al. (2024): reframes compression as token classification with a BERT-sized bidirectional encoder distilled from GPT-4, giving task-agnostic compression that is 3–6× faster than LLMLingua and reduces end-to-end latency by up to 2.9× at 2–5× compression.

A systematic study Li et al. (2023) cautions that, surprisingly, extractive compression “often outperforms” token-pruning methods and enables up to 10× compression with minimal degradation—i.e. pruning is not always the best hard method. Recent sentence-level extractive pruners build on this insight: Provence Chirkova et al. (2025) recasts context pruning as binary sequence labeling jointly trained with a reranker, giving a near-zero-cost RAG step faster than LLMLingua, and EXIT Hwang et al. (2025) classifies sentences context-aware to preserve inter-sentence dependencies while avoiding the token-by-token decoding latency of abstractive compressors. For coding agents specifically, SWE-Pruner Wang et al. (2026) learns a lightweight, structure-preserving task-aware classifier that prunes 23–54% of tokens on SWE-bench Verified while raising success rates, reaching up to 14.84× compression on single-turn LongCodeQA. A faithfulness-oriented alternative replaces token-level scoring with discourse structure: EDU compression Zhou et al. (2025a) parses text into a tree of Elementary Discourse Units anchored to source indices and linearizes only the query-relevant sub-trees, a “structure-then-select” route that preserves inter-unit dependencies for long-document and agent context.

4.2. Soft and Latent Compression

This family is model-modifying: rather than rewrite the prompt in natural language, it compacts context inside learned representations (soft tokens, memory slots, or KV values). Because it requires training or fine-tuning the model, it is not closed-API-compatible, which constrains where it can be deployed in practice.

Soft methods compress context into a small number of continuous vectors (soft tokens / memory slots / KV values), trading model-agnosticism for higher ratios Li et al. (2025b).

The founding soft-token line established that a prompt can be replaced by a few learned vectors—but each method modifies the model and compresses holistically (re-running whenever any span changes), so we treat it as a brief lead-in: Gisting Mu et al. (2023) (NeurIPS 2023) trains “gist” tokens for up to 26× compression and 40% FLOPs reduction on short instructions; AutoCompressor Chevalier et al. (2023) (EMNLP 2023) recursively folds segments into “summary vectors” over sequences up to 30,720 tokens; ICAE Ge et al. (2024) (ICLR 2024) pairs a LoRA encoder with a frozen decoder for $4\times$ compression at ∼1% extra parameters and up to $>7\times$ cached-slot speedup; and 500xCompressor Li et al. (2025c) carries information in per-layer KV values to reach 6–480× while retaining 62–73% of capability.

Recent directions: ratio, amortization, and convergence with KV. Newer soft methods push two axes—compression ratio and amortization—and increasingly blur the soft/KV boundary. Activation Beacon Zhang et al. (2024a) distills context into the key–value activations of special “beacon” tokens via a sliding-window stream, extending Llama-2-7B from 4K to 400K tokens at ∼2× faster inference and 8× lower KV-cache memory at near-lossless quality—soft-token and KV-cache compression converge when the learned soft tokens are themselves cached as KV states. LLoCO Tan et al. (2024) precomputes compressed representations plus per-document LoRA adapters offline, extending a 4K LLaMA2-7B to an effective 128K context with 30× fewer inference tokens and up to 7.62× speedup—paying the compression cost once and reusing it across many agent queries. For retrieval, xRAG Cheng et al. (2024) fuses a document’s dense-retrieval embedding into the LLM as a single soft token (frozen retriever and LLM), yielding a >10% average gain across six knowledge-intensive tasks at 3.53× fewer FLOPs, and COCOM Rau et al. (2025) compresses passages into a few context embeddings for up to 5.69× decoding speedup. A practical scalability limit is that most soft methods compress holistically, re-running the compressor whenever any span changes; CompLLM Berton et al. (2025) instead compresses fixed-size segments independently—linear in context length and cache-reusable—reporting up to 4× faster time-to-first-token and 50% smaller KV at long contexts, a property well-matched to incrementally-growing agent trajectories. Pushing the other way—scale rather than incrementality—LCLM (Latent Context Language Models) Li et al. (2026a) trains a small (0.6B) encoder to compress token sequences into latent embeddings for a larger (4B) decoder end-to-end on $>350$B tokens at fixed 1:4/1:8/1:16 ratios, reporting a better performance/speed/memory Pareto frontier than prior encoder–decoder soft compressors and signaling that soft compression is now being trained at pre-training scale. For agents specifically, the decisive property is amortization and segment-independence (LLoCO, CompLLM): because trajectories grow one event at a time, holistic compressors that must re-run on every change (gisting, ICAE) fit the re-prefill loop poorly, whereas cache-reusable segment compression does not.

The recurring limitation is that gisting/ICAE/AutoCompressor require modifying or fine-tuning the model, making them impractical for closed-source API agents—which is precisely why production general-purpose agents (Section 5) rely on hard compression and offloading instead. Autoencoding-free variants (e.g. contextual semantic anchors) Liu et al. (2025) are an active attempt to close this gap.

4.3. KV-Cache Compression and Serving-Time Controls

This family targets serving-time placement and compaction: it intervenes on the model’s internal key–value memory and on prefill compute during inference, rather than on the text of the prompt. Because these are deployment-time knobs on the inference stack, they are mostly relevant when self-hosting open-weight models; API-based agents cannot set them directly.

KV-cache methods compress the model’s internal key–value memory during inference, attacking the memory (and bandwidth) bottleneck of long contexts. They are largely training-free and hardware-compatible, and the literature has matured from per-token eviction into three structurally distinct mechanisms, surveyed comprehensively by Li et al. (2025a) (token-, model-, and system-level optimizations).

(i) Token eviction. StreamingLLM Xiao et al. (2024) (ICLR 2024) identifies the “attention sink” phenomenon—a few initial tokens absorb disproportionate attention—and keeps their KV plus a recency window, enabling stable modeling over 4M+ tokens with no fine-tuning (though it is content-agnostic and can discard semantically critical middle tokens). H2O Zhang et al. (2023) (NeurIPS 2023) keeps a small set of “Heavy Hitter” tokens carrying most attention mass plus recent tokens, improving throughput up to 29× and cutting KV memory up to $5\times$ without accuracy loss. SnapKV Li et al. (2024) (NeurIPS 2024) selects clustered important KV positions per head via a trailing prompt “observation window” and pooling, giving 3.6× faster generation and 8.2× memory savings at 16K context, fine-tuning-free. Rather than evict hard, a 2026 hybrid Meta-Soft Luo et al. (2026) synthesizes a few learnable soft tokens to stand in for removed entries and redistributes their attention mass into the retained cache—a soft-token/eviction blend that connects this section to Section 4.2.

(ii) Quantization. KIVI Liu et al. (2024b) (ICML 2024) quantizes the Key cache per-channel and the Value cache per-token to 2 bits with no fine-tuning, cutting peak memory ∼2.6× and raising throughput 2.35–3.47×—an axis orthogonal to eviction (fewer bits per entry rather than fewer entries).

(iii) Head/layer-structured allocation. Rather than a uniform budget, PyramidKV Cai et al. (2024) allocates layer-wise “pyramidal” budgets (more cache to lower layers), matching full-cache quality at ∼12% KV and beating H2O/SnapKV/StreamingLLM most at small budgets (64–128 tokens). DuoAttention Xiao et al. (2025) (ICLR 2025) splits heads into “retrieval heads” (full KV) and “streaming heads” (constant-length KV), reaching full accuracy at a 25–50% full-attention ratio and enabling 3.3M-token decoding on a single A100. Ada-KV Feng et al. (2025) provides the first head-wise adaptive budget allocation with a proven upper bound on attention-output error and plugs into SnapKV/PyramidKV for further gains on RULER and LongBench. RazorAttention Tang et al. (2025) (ICLR 2025) keeps a full cache only for retrieval heads, prunes remote tokens elsewhere, and recovers dropped content via a “compensation token”—compressing ∼70% of the KV cache (∼3×) with negligible degradation and FlashAttention compatibility. The common thread is that uniform per-token budgets are wasteful: allocating cache by head/layer importance is now the default, though all of these share the static-criticality weakness that Quest (below) exposes.

(iv) Low-rank / latent compression. A fourth, architectural mechanism compresses the KV cache into a low-rank latent rather than evicting or quantizing entries. The most production-impactful instance is Multi-head Latent Attention (MLA), introduced in DeepSeek-V2/V3, which projects keys and values into a small shared latent vector that is cached in place of full per-head KV, cutting KV memory by an order of magnitude while preserving quality DeepSeek-AI (2024); the broader low-rank line is catalogued as a distinct family by the KV-cache survey Li et al. (2025a). Unlike the post-hoc methods above, MLA is a model-design lever (chosen at pre-training), so it is available to agents only through the choice of base model rather than as a deployment-time knob.

Two caveats specific to agents. First, criticality is query-dependent: Quest Tang et al. (2024) (ICML 2024) shows that tokens deemed unimportant at prefill can become essential during decoding—precisely the failure mode static eviction triggers in multi-turn agents whose later queries revisit earlier context. Second, eviction interacts with safety: a 2025 study of five compression methods on Llama-3.1-8B and Qwen-2.5-14B finds compression does not degrade all instructions equally, and—using system-prompt leakage as a case study—shows leakage rises with compression and depends jointly on method, instruction order, and KV-eviction bias; because defense/guardrail instructions often sit in the (evictable) system prompt, aggressive KV compression can silently disable safety instructions Chen et al. (2025). A related but orthogonal lever is prefill-compute sparsification: MInference Jiang et al. (2024a) (NeurIPS 2024) exploits recurring dynamic sparse-attention patterns (A-shape, Vertical-Slash, Block-Sparse) to cut long-context prefill latency up to 10× at 1M tokens on an A100 with preserved accuracy, targeting the prefill bottleneck rather than KV memory.

Toward recoverable KV. Most of the above make permanent eviction decisions. KVzip Kim et al. (2025) (NeurIPS 2025, oral) instead scores each KV pair by how much it supports reconstructing the original context, producing a query-agnostic compressed cache reusable across future queries that cuts KV size 3–4× and decoding latency ∼2× with negligible loss—whereas query-aware eviction degrades even at a 90% budget under multi-query workloads. This directly matches the agent setting, where many unforeseen future queries are issued against the same history (Section 9). For agents, KV-cache compression is mostly relevant when self-hosting open-weight models; API-based agents cannot control it directly and instead rely on provider-side prompt caching (Section 5).

4.4. Retrieval and External-Memory Compression

The remaining, complementary family steps outside the model entirely. Where the natural-language summarization of Section 4.1 is destructive—once history is folded into a paraphrase the original spans are gone—retrieval and external-memory strategies are recoverable alternatives: they relocate context out of the window into a store that can be queried back, intervening at the placement and recovery stages of the lifecycle rather than discarding information outright. Compression rarely operates alone; the following strategies reduce or relocate context load while keeping a path back to the source.

4.4.1. Context Offloading (Filesystem as External Memory)

Rather than compress destructively, Manus treats “the file system as the ultimate context...unlimited in size, persistent by nature, and directly operable by the agent itself.” Its compression is “always designed to be restorable”: a web page’s content can be dropped as long as the URL is preserved, and a document’s content omitted if its path remains, so context shrinks Ji (2025, without permanent loss). The widely quoted “100:1” figure is best stated precisely: it is the average input-to-output token ratio Manus observed across millions of interactions (most spend is on processing context, not generation), which commentary frames as the motivation for restorable offloading rather than a measured lossless compression ratio Bhavsar (2025); the recoverability guarantee is what makes the offloading non-destructive. Anthropic’s Managed Agents generalize this to a durable session log queried via getEvents()—a context object living outside the model’s window Anthropic (2026a).

Offloading the tool surface, not just observations. A 2025 line of Anthropic engineering work offloads the often-overlooked cost of tool definitions and inter-tool data. Exposing MCP tool servers as code APIs—so the agent loads only the definitions it needs and filters results in a sandbox before they reach the model—reduced a representative workflow from 150,000 to 2,000 tokens (a 98.7% saving) Anthropic (2025b). A Tool Search Tool that discovers tools on demand rather than loading all definitions upfront cut tool-definition token overhead by ∼85% and left far more of the window for work—191,300 usable tokens with on-demand discovery vs. 122,800 when all definitions are preloaded, in Anthropic’s comparison—while raising tool-selection accuracy (49%→74% on one model), and Programmatic Tool Calling—orchestrating many tool calls inside one code block instead of round-tripping the model after each—cut average token usage on complex research tasks from 43,588 to 27,297 (37%) and eliminated 19+ inference passes when a task involved 20+ tool calls Anthropic (2025a).

4.4.2. Hierarchical/Virtual Memory (MemGPT, Letta)

Packer et al. (2023) (MemGPT) introduce virtual context management, an OS-inspired memory hierarchy: a fixed in-context “main” tier, plus “recall” (searchable history) and “archival” (vector storage) tiers, with the LLM itself issuing function calls to page information in and out—providing the “illusion” of unbounded context within a finite window. The successor framework Letta adds tiered core/recall/archival memory and background “sleeptime” agents for memory compaction Letta AI (2024). Cognitive taxonomies (episodic / semantic / procedural memory, per CoALA) underpin most modern agent-memory systems Sumers et al. (2024); Zhang et al. (2024d).

Production successors quantify what the virtual-memory abstraction buys. Mem0 Chhikara et al. (2025) dynamically extracts, consolidates, and retrieves salient facts, reporting 26% higher LLM-as-judge accuracy than a built-in memory baseline on the LoCoMo long-conversation benchmark while cutting p95 latency by 91% and token cost by over 90% versus passing the full conversation (its graph variant adds ∼2 points). MemoryOS Kang et al. (2025a) (EMNLP 2025, oral) makes the OS analogy explicit with three short/mid/long-term tiers and Store/Update/Retrieve/Response modules, reporting +49.11% F1 and +46.18% BLEU-1 on LoCoMo. Graph-structured memory is emerging as a competitive alternative to MemGPT’s vector tiers: Zep, built on the temporally-aware Graphiti knowledge-graph engine, surpasses MemGPT on Deep Memory Retrieval (94.8% vs. 93.4%) and improves LongMemEval accuracy up to 18.5% while reducing latency ∼90% versus a full-context baseline Rasmussen et al. (2025); A-MEM Xu et al. (2025) adds Zettelkasten-style note linking with “memory evolution” that updates old notes as new ones arrive. A 2025–2026 turn makes the memory operations themselves RL-learned rather than heuristic: Memory-R1 Yan et al. (2025) trains a manager to issue ADD/UPDATE/DELETE/NOOP over an external store (plus a learned distillation filter on retrieved memories) from only 152 QA pairs, and Mem-$\alpha$ Wang et al. (2025b) and MemSearcher Yuan et al. (2025) learn end-to-end when to store/update/retrieve during long interactions—the external-store analogue of the in-context, learned compaction policies of Section 6.

4.4.3. Retrieval-Augmented Generation (RAG) for Context Reduction

RAG injects only retrieved-relevant context, but retrieved passages are themselves long, motivating retrieval compression. RECOMP Xu et al. (2024) (ICLR 2024) compresses retrieved documents into textual summaries before integration, offering both an extractive compressor (select useful sentences) and an abstractive one (synthesize a summary), reducing cost and mitigating “lost in the middle.” Later work adds adaptivity: AdaComp Zhang et al. (2024b) predicts a query- and retrieval-quality-dependent compression rate, and AttnComp Luo et al. (2025) uses Top-P attention-weight selection to reach a 17× average compression ratio across five QA datasets (12.6× on HotpotQA to 23.9× on PopQA)—and is the only evaluated compressor that raises end-task accuracy (+1.9 points over the uncompressed baseline, while competitors lose ≥3) while cutting end-to-end latency to 49% of the uncompressed pipeline. (For RAG, soft compression is increasingly competitive: xRAG and COCOM collapse a passage to one or a few embeddings; see Section 4.2.) (Historically, the AutoGPT team removed external vector databases in late 2023, finding brute-force file search sufficient—an early signal that heavyweight retrieval is often overkill for real agent workloads Significant-Gravitas (AutoGPT) (2023).)

4.5. Sub-Agent Isolation

Sub-agent isolation is fundamentally a placement strategy: instead of compacting old context, it keeps detailed exploration out of the orchestrator’s context in the first place, returning only a distilled summary to the lead agent.

Isolating work in sub-agents keeps detailed exploration out of the orchestrator’s context. In Anthropic’s multi-agent research system, “each subagent might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000–2,000 tokens),” isolating search context from the lead agent; a Claude Opus 4 lead with Claude Sonnet 4 subagents “outperformed single-agent Claude Opus 4 by 90.2%” on their internal research eval, with token usage explaining ∼80% of performance variance Hadfield et al. (2025). This view is contested: Cognition (makers of Devin) argues against multi-agent designs for most tasks, advocating a single-threaded agent that shares full context and instead offloads long histories to a dedicated compression model that distills actions into key decisions and events—framing context isolation as a source of fragile, conflicting sub-decisions rather than a win Yan (2025). The cost/benefit of isolation is therefore task-dependent, not settled. OpenHands implements sub-agents as ordinary tools with isolated context All Hands AI (2025c); Hermes spawns isolated sub-agents and can “collapse multi-step pipelines into zero-context-cost turns via RPC” Nous Research (2026b). The trade-off is overhead: a single community estimate puts multi-agent coordination at ∼3.5× the token consumption of equivalent single-agent workflows because every handoff duplicates context Hayes (2026). The reconciling variable appears to be task decomposability: isolation pays off when subtasks are read-heavy and independent (open-ended research), whereas a single thread sharing full context wins when subtasks are tightly coupled and write-conflicting (iterative code editing)—which is why the strongest multi-agent results are reported on research-style evals and the strongest single-thread arguments come from coding agents. For long-form deep research specifically, Self-Manager Xu et al. (2026) pushes isolation to an OS-style abstraction, running multiple concurrent sub-threads under explicit “Thread Control Blocks” so no single window accumulates the full trajectory.

5. Production Patterns in General-Purpose Agents

The method families of Section 4 define the design space; the cleanest evidence for what actually works is in deployed general-purpose agents, where compression is the difference between a session that survives hundreds of tool calls and one that stalls against the context window. Stripped of benchmark framing, these systems converge on a small set of recurring patterns—trajectory compaction, budget guards, recoverable offloading, and cache-aware layouts—each an instantiation of the families just surveyed, which we organize below.

5.1. Trajectory Compaction in Deployed Agents

Hermes is the most precisely documented case Nous Research (2026a). Its context management is built on a pluggable ContextEngine; the default ContextCompressor runs two independent layers:

1.

Agent ContextCompressor (primary), fires at 50% of context using accurate API-reported token counts.

2.

Gateway session hygiene (safety net), fires at 85% using a rough character estimate, to catch sessions that ballooned between turns (e.g. overnight on Telegram/Discord).

The compressor runs a 4-phase algorithm: (1) cheaply prune old tool results >200 chars (“[Old tool output cleared...]”) with no LLM call; (2) determine boundaries protecting the first 3 messages (system + first exchange) and a token-budgeted tail (default ≥20 messages); (3) generate a structured summary (Goal / Constraints / Progress / Key Decisions / Relevant Files / Next Steps / Critical Context); (4) reassemble head + summary + tail, sanitizing orphaned tool-call pairs. On subsequent compressions it updates the previous summary rather than re-summarizing from scratch. A documented before/after example compacts 45 messages (∼95K tokens) to 25 messages (∼45K tokens). Separately, Hermes applies Anthropic prompt caching with a “system_and_3” breakpoint strategy that “reduces input token costs by ∼75% on multi-turn conversations” Nous Research (2026a).

Figure 7. A representative production trajectory-compaction design (Hermes-style). Two independent triggers—an accurate-token-count compressor at 50% and a character-estimate safety net at 85%—guard the window, and a four-phase rewrite prunes stale tool output, protects a head and a token-budgeted tail, regenerates a structured summary of the middle, and reassembles the transcript. Subsequent passes update the prior summary rather than re-summarizing from scratch.

Figure 7. A representative production trajectory-compaction design (Hermes-style). Two independent triggers—an accurate-token-count compressor at 50% and a character-estimate safety net at 85%—guard the window, and a four-phase rewrite prunes stale tool output, protects a head and a token-budgeted tail, regenerates a structured summary of the middle, and reassembles the transcript. Subsequent passes update the prior summary rather than re-summarizing from scratch.

Other deployed agents share the compaction skeleton but differ in where the rewrite decision lives. Claude Code/Anthropic combine server-side compaction, context trimming (removing stale tool results / thinking blocks, e.g. clear_tool_uses_20250919), the memory tool, and sub-agent isolation Anthropic (2026a); All Hands AI (2025c). OpenHands offers nine pluggable condensers composable into pipelines, operating at the view level (inserting markers, never mutating the raw event stream) so sessions remain replayable; its condenser cut API cost ∼50% in deployment and held latency near 8 s vs. 16 s for the uncompressed baseline at iteration 100, resolving 200 vs. 203 SWE-bench Verified instances All Hands AI (2025a); Smith (2025); Wang et al. (2025a); Rombaut (2026). Cline notably delegates the compaction decision to the LLM via a condense tool and re-injects a task checklist (“Focus Chain”) every few messages to resist drift Rombaut (2026); Baumann (2025).

5.2. Budget Guards and Mundane Controls

OpenClaw does not use a tokenizer library; it estimates with a 4-chars≈1-token heuristic (2 chars/token for token-dense tool output) for pre-call guards, pruning, and adaptive sizing centminmod (2026). Its compression-relevant levers include: truncating oversized tool results (max ∼30% of the context window per result, hard cap 400,000 chars); a /compact summarization command; contextPruning (e.g. cache-ttl mode); on-demand MEMORY.md/memory-search rather than always-injecting long files; bootstrap-injection caps (bootstrapMaxChars, bootstrapTotalMaxChars); and Anthropic prompt caching (cache reads cost ∼10% of normal input tokens) auto-enabled for API-key sessions centminmod (2026); OpenClaw (2026c). A community “Context Manager” skill performs AI-summary-and-replace at 70–80% usage, backing up the original JSONL transcript to memory/compressed/ before resetting the session plgonzalezrx8 (2026), and a third-party “Context Optimizer” advertises 40–60% token reduction AtlasPA (2026). Empirically, the dominant cost driver is unbounded history, and the most effective mitigations reported by the community are mundane: model routing (cheap models for heartbeats), aggressive session resets/compaction, and cache-TTL alignment; community write-ups consistently claim (unaudited) 80–95% cost reductions from these combined LaoZhang AI (2026); Hayes (2026). A structural caveat: lowering historyLimit can increase tokens, because a too-short window pushes the agent to issue more memory_search/grep/read tool calls, each adding tokens Liu (2026).

The same mundane controls recur in the broader Claw ecosystem. ClawMobile inherits OpenClaw’s memory mechanism, embedding lightweight execution preferences and reusable task patterns into context or the retrieval path Du et al. (2026); MetaClaw injects distilled procedural skills into the prompt each round without weight updates Ji et al. (2026).

5.3. Recoverable Context by Design

A second axis separates agents that discard compacted content from those that keep it restorable. Manus treats the filesystem as externalized context, offloading state to files and re-reading on demand rather than holding everything in the window Ji (2025); OpenHands’ view-level condensers insert markers over a never-mutated raw event stream, so any session remains replayable from the underlying log All Hands AI (2025a); and MemGPT/Letta organize memory as an explicit hierarchy with paged in/out movement between a small in-context working set and larger external tiers Packer et al. (2023); Letta AI (2024). The shared idea is that compaction need not be lossy at the system level even when it is lossy in-window: the dropped tokens live in a file, an event log, or an external memory tier that the agent can page back. Section 4 gave the method detail for these mechanisms; here the point is operational—production agents lean on recoverability to make aggressive in-window compaction safe.

5.4. Prompt-Cache-Aware Layouts

Because providers bill cache reads at a fraction of fresh input tokens, the most token-efficient production layouts are organized around what the cache can keep warm. The recurring pattern is a stable system head, a protected recent tail, and a middle that is summarized by update rather than rewrite—so a single compaction does not invalidate the cached prefix on every turn. Hermes encodes this directly: its “system_and_3” breakpoint strategy caches the system prompt plus the first three messages and reports ∼75% input-cost reduction on multi-turn conversations, while its compressor updates the prior summary instead of regenerating it so the protected head stays cache-stable Nous Research (2026a). OpenClaw aligns its contextPruning with a cache-ttl mode and auto-enables Anthropic prompt caching for API-key sessions, where cache reads cost ∼10% of normal input tokens centminmod (2026). Both rest on the provider primitive that cached prefixes are billed at a steep discount and survive only as long as the prefix is left byte-stable across turns Anthropic (2024).

5.5. Production Agents at a Glance

The recurring pattern across these systems is trajectory compaction + tool-output pruning, typically paired with provider-side prompt caching; they differ most in recoverability—OpenHands’ view-level condensers and Manus’ filesystem offloading keep a replayable/restorable record, whereas pure summarization (Hermes, Cognition) does not—and in how far they push offloading (Manus, Claude Code, and OpenClaw relocate content to files or tools) versus sub-agent isolation (Hermes, Claude Code, OpenHands, and Manus spawn isolated sub-agents). Rather than a coarse per-family yes/no inventory, Table 3 makes the choices that actually distinguish agent designs comparable—trigger, protected span, compaction target, recovery source, and cache interaction. A finding worth stating explicitly is what the table cannot fill in: none of the surveyed production agents documents provenance tracking or privilege enforcement on compacted content—an empty cell in the design space that corroborates the lifecycle discussion of Section 3, where governance is a normative rather than observed stage, and the security concerns of Section 9.

6. Learned, Agent-Native Compression: The Frontier

The production designs above treat compression as an external heuristic scaffold. The fastest-moving line of 2025–2026 instead folds context management into the agent’s own learned policy, trained end-to-end with reinforcement learning. The common pattern is to make the compaction step an action: when the working context crosses a budget threshold, the agent (or a learned manager) emits a summary / memory-write / eviction, the window is reset to that compact state, and exploration resumes—so the effective horizon is the working window multiplied by the number of compactions. We group the cluster by what is learned; Figure 3 places it in time and Figure 8 situates its reductions against the other families. Table 4 maps each cluster back to the family and lifecycle axes of Section 3: every method here compresses in the hard/natural-language token space at the trajectory or memory level, and acts mainly at the compaction and placement stages, so the frontier is a learned-policy traversal of the same lifecycle rather than a fifth method family (we use “hard,” “natural-language,” and “token-space” interchangeably).

6.1. Compression as Action

Folding sub-trajectories. Context-Folding  Sun et al. (2025) trains an agent (via FoldGRPO) to branch into a sub-trajectory and “fold” it to a summary on completion, keeping the active context ∼10× smaller than ReAct while matching or beating it on Deep Research and SWE tasks; AgentFold Ye et al. (2025) adds multi-scale folding (granular condensation vs. deep consolidation), holding context to ∼7K tokens after 100 turns (scaling to 500) and reaching 36.2% on BrowseComp.

Summarization-augmented policy optimization (SUPO). SUPO Lu et al. (2025) puts the summarization-as-policy idea on a principled training footing. It formalizes a summarization-augmented MDP: whenever the accumulated context exceeds a threshold L, a summarization prompt is appended, the agent’s own policy generates a task-relevant summary, and the working context is reset to the initial prompt plus that summary (effective length $L_{\mathrm{eff}}=L\times (S+1)$ for S summarizations). The key result is a policy-gradient decomposition that splits a long rollout’s gradient into a sum over the summarized sub-trajectories, so standard GRPO/PPO-style infrastructure trains tool-use and summarization end-to-end on horizons far longer than the working window—no separately trained summarizer required. Three design choices carry the method: trajectory management across sub-trajectories; group-relative advantage estimation across the whole rollout group (computing advantage within a single trajectory costs $2.4$ points on CodeGym); and an overlong-trajectory mask that stabilizes optimization (removing it collapses gains from $+3.2$ to $+0.8$ on CodeGym and from $+14.0$ to $+5.0$ on BrowseComp-Plus). Empirically, SUPO reaches $47.7\%$ vs. $44.5\%$ (vanilla multi-turn GRPO) on CodeGym using only a 4K working window for a 32K effective horizon ($8\times$), and $53.0\%$ vs. $39.0\%$ on BrowseComp-Plus at a fixed 64K window; crucially, allowing more summarization rounds at test time than were seen in training pushes accuracy further to $60.0\%$. SUPO subsumes segment-overwrite memory (MemAgent Yu et al. (2025)) as a special case and, unlike MEM1’s fixed-size state (Section 6.2, below), provably scales training beyond the reliable context window.

Summarization for deep web-search agents (ReSum). ReSumWu et al. (2025) specializes periodic summarization to long-horizon search, where context blow-up is most acute. As a plug-and-play ReAct replacement, when the trajectory nears the context budget it invokes a purpose-trained summary tool—ReSumTool-30B, an SFT of Qwen3-30B on <Conversation,Summary> pairs—that condenses the think/search/observe history into a structured “reasoning state” recording verified evidence plus an explicit list of remaining information gaps and next steps; the working history is reset to this compact state and exploration resumes from it. To teach the policy to reason well from summaries, ReSum-GRPO adapts GRPO with segmented-trajectory training and advantage broadcasting (the trajectory-level advantage is broadcast across all segments for long-horizon credit assignment). Training-free, ReSum already adds $+4.5\%$ over ReAct on average; after ReSum-GRPO the gain reaches $+8.2\%$ (GAIA Pass@1 $45.0\to 47.3\to 48.5$, BrowseComp-en $12.8\to 16.0\to 18.3$, BrowseComp-zh $23.9\to 24.1\to 33.3$), with the resulting agent (WebResummer-30B) trained on only ∼1K samples.

6.2. Memory as Action

Constant-memory consolidation (MEM1). MEM1 Zhou et al. (2025b) is the founding instance of making memory consolidation a learned action. Rather than letting the transcript grow, the agent maintains a single constant-size “internal state” that it rewrites every turn—integrating the prior state with the new observation while discarding redundant or now-irrelevant content—so memory consolidation and reasoning are optimized jointly and context no longer scales with the horizon. Trained end-to-end with RL, MEM1-7B improves task performance $3.5\times$ while using $3.7\times$ less memory than a $2\times$-larger Qwen2.5-14B baseline on a 16-objective multi-hop QA task, and generalizes beyond its training horizon across retrieval QA, open-domain web QA, and multi-turn web shopping. Its limitation—a fixed-size state—is exactly what the next two methods relax.

Memory writes as learned actions. A parallel strand turns explicit memory edits into RL actions. Memory-as-Action Zhang et al. (2025b) treats working-memory deletes/inserts as actions, letting a 14B model match $16\times$-larger models while cutting average context length $51\%$. MemPO Li et al. (2026b) introduces a <mem> action (alongside <think> and tool calls) so the agent writes its own step-wise memory summary and is prompted only with the most recent <mem> state; it is trained with a dual advantage—a trajectory-level signal (answer-correctness and format) plus a memory-level signal that scores whether the memory content alone makes the correct answer likely—reporting $+25.98$ F1 over its base model with 67–$73\%$ fewer tokens. AgeMem Yu et al. (2026) generalizes this to a full learned memory-op toolset (Add/Update/Delete/Retrieve/Summarize/Discard) trained with three-stage progressive RL.

6.3. Credit Assignment for Compression Decisions

Less-entangled credit for memory writes (HiMPO). These designs leave a subtle training problem. In a compressed-memory agent that rewrites a <mem> state each round, a single memory write is causally entangled with everything that follows it—tool calls, noisy observations, and downstream reasoning—so a trajectory-level reward can punish a faithful write for an unrelated later failure, or credit a lossy write that merely preceded success. HiMPO Yan et al. (2026) isolates a memory-specific advantage applied only to the <mem> tokens, built from a local counterfactual utility (how much rendering the identical pre-write state with the new rather than the previous memory raises the likelihood of the target outcome, independent of the later rollout) gated by a hindsight relevance filter (a retrospective gate that attenuates this credit when the realized outcome does not bear it out). Over MemPO’s combined-reward design, HiMPO reports sharper memory-drop attribution and gains of $+4.7$–$4.9$ F1/EM and $+2.4$–$2.8$ points on BrowseComp-Plus/FRAMES at essentially unchanged token budget, with an ablation identifying the retrospective gate as the component that does the disentangling.

6.4. Budget-Conditioned Managers

Budget-aware and externally-managed variants. ContextBudget (BACM-RL) Wu et al. (2026) conditions on the remaining token budget and chooses Null/Partial/Full compression under a curriculum that tightens the budget $8\mathrm{K}\to 4\mathrm{K}$, beating MEM1 by $2.43\times$ on 32-objective QA and degrading gracefully as budgets shrink. A distinct design point keeps the agent frozen and learns an external manager: AdaCoM Yi et al. (2026) RL-trains a separate manager that edits a frozen agent’s context (averaging $+39\%$ over ReAct on BrowseComp-Plus and transferring across agent pairs), and ContextCurator Li et al. (2026c) trains a lightweight curator with multi-turn GRPO that rewrites working memory for a frozen executor, cutting DeepSearch context ∼8× ($46.7\mathrm{K}\to 6.6\mathrm{K}$ tokens) while raising success ($53.9\%\to 57.1\%$). These sharpen the external-scaffold vs. learned-policy axis (Section 3): they are learned but external, occupying the middle ground between training-free ACON and fully in-policy folding. Finally, RE-TRAC Zhu et al. (2026) compresses across trajectories, recursively distilling each search attempt into a structured state that seeds the next ($+15$–$20\%$ on BrowseComp with monotonically falling tool calls and tokens).

6.5. Why the Frontier is Agent-Specific

Why now. This frontier is driven by long-horizon deep-research / web-search agents, where context blow-up is most acute and the bottleneck is increasingly efficiency rather than raw accuracy—e.g. Chen et al. (2026) cut average reasoning steps on BrowseComp by $70.7\%$ by acquiring evidence in parallel within a constrained budget. The unifying shift is from compression-as-scaffold to compression-as-policy, operationalizing Anthropic’s framing of compaction as managing a finite “attention budget” against the $n^2$ growth of pairwise token relationships Anthropic (2025c). Most of these methods compress natural-language summaries/memory at the trajectory level and are therefore orthogonal to (and combinable with) the KV/soft compression of Section 4.2 and Section 4.3.

6.6. Comparison of Methods

Table 5 and Table 6 consolidate reported quantitative results—single-call prompt/soft/KV compression and agent-level trajectory/memory compression, respectively—and Figure 8 plots the reported reduction factors across families on a shared linear scale (off-scale values are capped and annotated with their true value). Numbers are as-reported by the cited sources under heterogeneous settings and are not directly comparable across rows; treat them as order-of-magnitude evidence. Nor are they all instances of the compression ratio $\rho =|C_t|/|{\tilde{C}}_t|$ defined in Section 3: the reported numbers mix prompt-token ratios, KV-memory and throughput multipliers, cost reductions, and effective-context extensions, and the caption and Table 5 and Table 6 state which quantity each number measures. The landscape shows a clear pattern: soft-token and recoverable-offload methods reach the most extreme ratios (but are lossy or recoverable-by-pointer), whereas KV-cache and the learned, agent-native methods occupy a more moderate, accuracy-preserving regime.

7. Evaluation: From Compression Ratio to Agent Utility

Compression ratio and single-call accuracy are the wrong yardsticks for agent compression, and reporting either in isolation hides the trade-off that actually matters. Most agent-compression results come from small instance counts (Focus: $N=5$) or single scaffolds, and savings are highly task-dependent (exploration-heavy tasks benefit; iterative-refinement tasks can see compression overhead) Verma (2026). The problem is compounded by which long-context benchmark is used: HELMET Yen et al. (2025) shows that synthetic needle-in-a-haystack (NIAH) scores and arbitrary task subsets do not predict downstream behavior and can reorder model rankings, so a method that wins on one probe may lose on application-centric suites such as LongBench v2 Bai et al. (2024), ∞Bench Zhang et al. (2024c), or BABILong Kuratov et al. (2024) (the NIAH test itself originates as an open-source pressure probe Kamradt (2023)). Crucially, long-context benchmarks score accuracy in isolation while agent-compression results report tokens in isolation; almost none score both. LoCoBench-Agent Qiu et al. (2025) is a rare exception, converting 8,000 software-engineering scenarios into interactive agent environments (10K–1M tokens, 8 tools) and reporting nine metrics split into comprehension and efficiency dimensions, surfacing an explicit comprehension–efficiency trade-off. Standardized agent-centric benchmarks that jointly report accuracy and peak/total tokens (as ACON and LoCoBench-Agent attempt) are needed Kang et al. (2025b); Qiu et al. (2025).

Metrics. A useful evaluation of agent compression must report a vector of quantities, not a single ratio: how well the task was done, how much context was billed at the peak and over the whole session, how much extra work (output and tool calls) the compression induced, how it interacts with the prompt cache, and whether the compacted context remains recoverable, faithful, and secure. Table 7 enumerates the reporting metrics we advocate, what each captures, and why it is needed. The set is deliberately full-session: peak context tokens bound the largest prefill the agent ever pays for, total session input tokens capture the cumulative repeated-prefill bill, and the cache and recovery rows capture failure modes (cache-miss spikes, delayed-relevance retrieval failures) that single-call accuracy cannot see.

Benchmarks. Existing benchmark families each measure a real property but none measures full-session agent utility on its own; Table 8 pairs each family with what it misses and the agent-specific addition required to make it diagnostic. Needle and retrieval probes measure positional recall but are weak predictors of tool-using agents; LongBench-style QA measures long-context answer quality but is single-shot rather than looped; SWE and web-agent suites measure task completion but are scaffold-dependent; and memory benchmarks measure multi-session recall but may ignore cost.

We frame these metrics and benchmark additions as a reporting protocol meant to be adopted, rather than an afterthought—a usable measurement guide alongside the map of the literature.

8. Design Guidelines and Recommendations

For practitioners building or operating general-purpose agents, we recommend a staged approach, ordered by effort-to-impact and aligned with the lifecycle taxonomy of Section 3. The ordering walks the lifecycle from cheapest, most reversible intervention to most invasive: first reduce admission—cap tool output, shorten tool schemas, and route cheap background turns to small models so fewer tokens ever enter the context; then stabilize layout for caching—fix a protected head, a compact middle, and a recent tail so the prompt prefix stays cache-stable; then choose recoverable placement—offload large artifacts to files, the raw transcript, or a retrieval store so dropped content can be re-fetched on demand; then add destructive compaction only when recovery is genuinely not needed; and finally consider learned policies or KV/soft compression when model access and the workload justify the added machinery. The staged recommendations below instantiate this progression.

1.

First, cheap structural fixes (highest ROI). Route low-value turns (heartbeats, status checks) to cheap models; reset sessions on task boundaries; truncate large tool outputs; keep always-injected bootstrap/system files short. Community write-ups report (unaudited) 80–95% cost reductions from these alone LaoZhang AI (2026); Hayes (2026); we present this ordering as a practitioner heuristic pending controlled measurement, not an established result. Threshold to revisit: if context still exceeds ∼50% of the window mid-task.

2.

Then, enable trajectory compaction with caching. Adopt an OpenHands-/Hermes-style condenser that fires at 50–80% utilization, protects the system prompt and a recent tail, summarizes the middle with a structured template, and updates (not re-creates) prior summaries. Align pruning TTL with prompt-cache TTL to avoid cache-miss spikes Nous Research (2026a); All Hands AI (2025a); Wu (2026). Benchmark to track: task success rate and peak tokens jointly; back off compression aggressiveness if success drops >2–3 points.

3.

Add recoverable offloading for large artifacts. Drop verbose content (web pages, file dumps) from context while keeping URLs/paths so it can be re-fetched; let the agent maintain a todo.md/scratchpad Ji (2025); Bhavsar (2025). Prefer this over destructive truncation whenever the artifact may be needed later.

4.

Isolate exploration in sub-agents for research/codebase-exploration tasks, returning only 1–2K-token summaries to the orchestrator—accepting the (community-estimated) ∼3.5× coordination overhead only when the task genuinely parallelizes Hadfield et al. (2025); Hayes (2026).

5.

Consider RAG/extractive compression (RECOMP-style) when context is dominated by retrievable documents; prefer extractive over heavy pruning given evidence that extraction often wins Xu et al. (2024); Li et al. (2023).

6.

Reserve soft/embedding and KV-cache compression for self-hosted open-weight deployments. Gisting/ICAE/500x and H2O/SnapKV give the largest ratios but need model access; they are not actionable for closed-API agents, which should instead rely on provider prompt caching and the above Mu et al. (2023); Ge et al. (2024); Zhang et al. (2023).

What would change these recommendations: (a) if model providers ship cheap, recoverable server-side compaction (as Anthropic Managed Agents trend toward), steps 2–3 collapse into the platform; (b) if context windows become effectively free of “context rot” (no degradation at 1M tokens), aggressive compression becomes harmful and should be relaxed; (c) if provenance-preserving compression matures, security concerns (step gating) ease.

9. Open Challenges

Cache-compression co-design. Compaction invalidates prompt-cache prefixes; misaligned pruning/cache TTLs cause expensive cache-miss spikes that can cost more than the tokens saved Wu (2026); centminmod (2026). As quantified in Section 2, cache reads are billed at only ∼0.1× input, so a compaction event that rewrites the cached prefix forfeits that discount and naive condensation can be a net cost increase Anthropic (2024). Co-designing compression with caching is an underexplored systems problem.

Recoverability and delayed relevance. Lossy compaction discards tokens whose future relevance is unknown. Anthropic notes “irreversible decisions to selectively retain or discard context can lead to failures” Anthropic (2026a), and lossy autocompaction can discard a session’s hard-won understanding wholesale—the 98% reduction documented in Section 2 Santoni (2026). Recoverable offloading and externalized context objects are the leading mitigation Ji (2025); Anthropic (2026a); inside the KV cache, the analogous move is reconstruction-scored compression—KVzip produces a query-agnostic, reusable compressed cache rather than query-conditioned eviction, which degrades under the many-unforeseen-queries workload typical of agents Kim et al. (2025). Recoverability is thus emerging as a first-class design axis, not an afterthought.

Faithfulness, drift, and provenance. Repeated summarization subtly rewords and de-prioritizes the original task, causing reasoning to diverge Zylos Research (2026). Agentic Context Engineering (ACE) names the two failure modes precisely: “brevity bias,” where successive summaries drop domain detail in favor of concise prose, and “context collapse,” where iterative rewriting erodes information over time. ACE counters both by treating context as an incrementally-updated “playbook” edited via structured delta updates that preserve provenance, reporting +10.6% on agent (AppWorld) and +8.6% on finance benchmarks with lower adaptation latency Zhang et al. (2025a). Iterative update-style summaries (Hermes) and structurally lossless trimming (DAG-based virtualization) are kindred responses Nous Research (2026a); Santoni (2026). Compression schemes that preserve provenance—tracking which tokens are derived, summarized, or verbatim—are therefore a prerequisite for trustworthy long-horizon agents, not merely a faithfulness nicety.

Security and privilege boundaries. Compression interacts with safety. KV-cache eviction can bias toward dropping system/defense instructions, enabling prompt leakage Chen et al. (2025). In the Claw ecosystem, ClawWorm demonstrates on an isolated OpenClaw testbed that flat context trust and unconditional configuration loading can enable persistent, self-propagating compromise Zhang et al. (2026); scheduled heartbeat turns add an unattended mechanism for reading or updating persistent state OpenClaw (2026b). Compression schemes that preserve provenance/privilege boundaries are a future need.

Benchmarking full-session utility. Long-context benchmarks score accuracy in isolation while agent-compression results report tokens in isolation; standardized agent-centric benchmarks that jointly report accuracy and peak/total tokens over a full session are still missing Kang et al. (2025b); Qiu et al. (2025). Section 7 treats this gap in detail.

Training stable compression policies. The most promising direction unifies threads above: agent-controlled (Focus), guideline-optimized and distilled (ACON), recoverable (Manus, KVzip), and provenance-aware (ACE) compression—ideally learned end-to-end. The clearest signal of this convergence is the move from compression-as-scaffold to compression-as-policy: Context-Folding, AgentFold, and Memory-as-Action train the agent itself to decide what to fold, consolidate, or delete, reporting order-of-magnitude active-context reductions at parity or better accuracy Sun et al. (2025); Ye et al. (2025); Zhang et al. (2025b). The open problem is training such policies stably: the compression decision is a discrete, high-variance action whose reward (downstream task success) is delayed and confounded with the base agent’s competence. 2026 commentary correspondingly suggests context-window size is plateauing while focus shifts to “hybrid compression+caching, and memory-augmented architectures” Zylos Research (2026).