Submitted:
22 October 2026
Posted:
27 October 2026
You are already at the latest version
Abstract
Agent delegation can disclose sensitive information across organizational boundaries, while centralized logging can create a second repository of sensitive content. We study the separation between metadata-based auditing and local release enforcement. An experimental A2A metadata interface exposes labels, identities and hashes rather than raw messages; a separate structured gate checks authorization, case relevance and exact source support. These are scoped checks, not guarantees of semantic confidentiality. On 20 authored tasks with shared frozen model proposals, the full gate preserves eight benign completions, removes two forbidden-marker and four unsupported-value releases, and returns unknown on nine tasks. In action-prefix replay of 1,132 archived AgentDojo trajectories, destination binding reduces native attack success from 165/808 to 0/808 while clean completion falls from 198/324 to 179/324. A separate frozen live study compares eight methods on 32 development-disjoint tasks using one 14B model and one seed. Ungated execution completes 10/32 tasks with one attack success in 16 attacked tasks; defended variants complete 0–12 tasks with zero observed attacks. Sparse attacks and wide paired intervals preclude a defense ranking. We expose the observation boundaries, failure cases and cost accounting needed to interpret these results without conflating non-disclosure, task failure and privacy.
Keywords:
multi-agent systems
; privacy
; metadata auditing
; release enforcement
; prompt injection
1. Introduction
A delegated task can carry private facts across principals even when each individual tool call is syntactically valid. A scheduling assistant may reveal a visit time while a second assistant supplies a clinic location; a recipient can combine the two. Moving all messages into a central monitoring service makes inspection easier but creates another concentration of sensitive data. For organizations deploying agents across workspaces, the practical problem is therefore twofold: reduce the content collected by governance infrastructure and control what business actions actually release.
We examine a deliberately limited approach (Figure 1). Local components label handoffs; a center checks the reported relationships without receiving raw message text. A distinct release adapter checks proposed fields against current authority and permitted source evidence before a fixed renderer emits them. We call the observation boundary center-blind: it excludes raw text from the specified report schema, not information about the text. Categories, identities, graph links and equality hashes can themselves leak. Nor can a center establish the truth or completeness of an endpoint’s labels merely by authenticating a report.
Contributions
An explicit observation contract. We separate data subject, owning principal and acting agent, specifying what local labeling reports and what the center cannot verify. A scoped enforcement contract. A separate release check distinguishes authorization, case relevance and exact source support, preserving unknown outcomes when evidence is missing. A paired evaluation of release tradeoffs. Three distinct regimes separate mechanism checks, archived blocking and live execution, reporting completed tasks and costs alongside native attack outcomes. Replay measures blocking rather than recovery; sparse live attacks and wide paired intervals do not support a defense ranking.
Practical Privacy Relevance
The motivating applications are delegated business workflows spanning mail, messaging, retrieval and document updates. A governed application can supply policy labels and enforce a release boundary without changing its underlying planner. Our experiments use controlled business records and benchmark tools, not customer accounts. They test specific ways in which sensitive content may leave a workflow; they do not estimate population-level leakage rates or establish production readiness. There is no federated model-training contribution.
2. Problem and Evidence Boundary
The unit of analysis is an agent-to-agent handoff across a data-governance boundary. We distinguish the data subject whom a fact describes, the owning principal controlling the resource, and the acting agent that sends or receives it. Shared ownership does not automatically authorize all purposes, recipients or fields.
A recipient may retain and combine messages, a forwarder may exceed its scope, and a local reporter may omit events or mislabel their content. Our software baseline trusts the local observation and labeling path for semantic detection; it does not solve malicious endpoints or unmediated communication. Report MACs authenticate an issuer’s statement, not its truth. Replay prevention, completeness and key isolation are separate obligations.
What the Center Can observe
The center receives labels, routing identities, provenance and content hashes, not raw message text. This reduces centralized content collection but still reveals metadata and equality patterns. Low-entropy hashes may permit dictionary attacks. A missing alert therefore cannot certify confidentiality: the report may be incomplete, its labels wrong, or the relevant inference outside the model.
3. A Scoped Audit and Release Interface
3.1. Metadata Audit
The experimental a2a.privacy/v1 extension attaches governance fields to A2A Parts: subject, owner, sensitivity, categories, inferred categories, allowed recipients, purpose, hop budget and provenance. It uses A2A’s metadata mechanism; it is not an adopted A2A standard. A local component derives labels and a central graph checks four relationships: disclosure to an unauthorized principal, incompatible purpose, forwarding beyond a hop budget, and accumulated inference across messages.
Graph Scope
The detector groups supplied edges by recipient principal and data subject, deduplicates equal content hashes, and combines inferred-category tags. A session can submit multiple messages together. This does not establish detection across independently retained reports, truthful subject resolution, or an adversarial recipient’s actual ability to infer an attribute. Graph pseudonyms preserve linkability needed by the check and are not graph anonymity. The three empirical regimes below do not validate a complete end-to-end composition of this audit with every release control.
For the inference check we use an explicit odds model, , and flag a posterior gain exceeding a chosen threshold. Its likelihood ratios and conditional-independence assumption are modeling choices, not calibrated guarantees about real recipients. Accurate labels alone do not validate these assumptions. A posterior alert is conditional on the supplied labels and model, rather than a measured recipient attack.
3.2. Release Checks and Their Implementation
An audit report must distinguish three claims: authorization at release (A), relevance to the requested case and purpose (R), and support by permitted source evidence (S). Authorized context can belong to the wrong case, and relevant context can still lead to an unsupported assertion. Checking A does not establish . Missing evidence produces unknown, not a positive safety verdict or a fact about the world.
We adopt the separation of model proposals from trusted authority found in capability and information-flow systems, including CaMeL [1], FlowSeal [2] and FAVA [3]. Our question is which scoped claims a content-blind center can justify when evidence is incomplete. We do not reproduce those systems or claim empirical superiority over them.
Implemented Structured Gate
Each proposed field carries a source identifier, version, field name and value. A trusted local adapter supplies workspace, case, recipient, purpose, source fields and authorization state. The full gate requires current recipient access, matching workspace and case, an allowed field-purpose pair, matching version and exact equality to the source value. It rejects permission overrides and extra free text. Missing authority or support returns unknown; established mismatches return deny. Only admitted fields reach a fixed renderer, and the entire proposal is withheld if any tuple fails.
An experimental PostgreSQL adapter serializes this check with a business-side mailbox commit. That boundary does not guarantee external delivery or remove information already in model memory. Source facts can be wrong, exact matching rejects valid paraphrases, and individually allowed fields can enable inference. A minimal center-facing summary exposes status and fixed reason codes; it is neither differentially private nor independently verifiable semantic evidence.
What Remains Proposed
General free-text support verification and a complete composition of semantic auditing with release enforcement remain unvalidated. The current gate is a restricted executable check, not a natural-language entailment system. The receipt interface binds identifiers and an epoch but does not implement every proposed purpose, version and support obligation. The structured gate and the AgentDojo action controls below are distinct implementations; the live controls do not validate exact-field support for arbitrary text.
4. Evaluation: Privacy, Utility and Cost
We organize the evidence by what each execution mode can establish. Authored structured tasks test mechanism behavior; external replay measures intervention on archived actions; live tasks measure execution under fixed budgets. We do not pool these units or equate native attack failure with semantic privacy.
Table 1.
Evidence units and interpretation boundaries. Repeated controls are paired observations, not independent tasks.
Table 1.
Evidence units and interpretation boundaries. Repeated controls are paired observations, not independent tasks.
| Evidence | Scale | Model coverage | What the regime tests |
|---|---|---|---|
| Structured release | 20 authored tasks, 5 gates | Pinned Qwen2.5-3B | Shared proposals; exact-field checks |
| External replay | 1,132 traces, 81 underlying tasks | Archived provider/model runs | Prefix blocking; no recovery |
| Live execution | 32 tasks, 8 methods, 256 episodes | Pinned Qwen2.5-14B, one seed | Bounded action execution and costs |
4.1. Structured Release
A frozen gate and scorer evaluate 20 new authored tasks using one pinned Qwen2.5-3B proposal per task. Every gate sees the same proposal. The full gate completes all eight benign requests, reduces literal forbidden-marker releases from two to zero and unsupported-value releases from four to zero. Deny-all also removes these releases but completes no benign request. Scope checking removes the two marker failures while leaving the four unsupported values; exact source matching removes those values.
The full gate returns unknown on nine tasks: two schema failures, three cases without authority and four unsupported values. Three further requests are denied. Two malformed proposals are rejected before policy evaluation, limiting any revocation-specific interpretation. Source errors and valid paraphrases remain counterexamples. This small authored transfer check establishes neither broad semantic correctness nor production performance. The full gate has eight allow, three deny and nine unknown decisions; these categories exhaust the 20 tasks.
4.2. External Replay
We freeze 1,132 AgentDojo [4] archived trajectories: 324 clean and 808 attacked, covering 81 underlying tasks. Five controls replay the same actions using unchanged native task and attack predicates. At the first blocked egress action, replay terminates; the original later answer is removed. This tests a destination-authority adapter, not semantic message support or how an agent recovers after a refusal.
Combined intent and destination checks reduce native exfiltration success from 165/808 to 0/808. Clean utility falls from 198/324 to 179/324, a paired change of percentage points (task-cluster bootstrap 95% interval ). Attacked utility falls from 383/808 to 316/808, a reduction of 8.3 percentage points. The 95% interval for the change is . Destination-only checking produces the same counts; this experiment supplies no measurable benefit for the additional intent rule.
Figure 2.
External replay retains all selected trajectories and separates native attack success from task completion. The deny-egress arm has one clean and one attacked checker exception, retained as unresolved. Unresolved checker exceptions are not counted as a privacy success. The plotting implementation was prepared with AI assistance.
Figure 2.
External replay retains all selected trajectories and separates native attack success from task completion. The deny-egress arm has one clean and one attacked checker exception, retained as unresolved. Unresolved checker exceptions are not counted as a privacy success. The plotting implementation was prepared with AI assistance.

Losses concentrate in Slack and include legitimate implicit destinations that literal binding cannot resolve. The archive also has checker-version discrepancies: we retain and audit them instead of replacing the native predicates or selecting only successful reconstructions. These losses qualify the eight-of-eight benign retention in the authored set.
4.3. Controls and Statistical Units
The eight live methods share the model, base instruction and total resource caps. No gate executes proposed actions. Gate applies the combined destination/intent policy and returns refusals to the model; scope additionally checks explicit operation binding for webpage writes. Receipt binds an authorized action to the task and an epoch before consumption. Deny-egress blocks outgoing actions. Tool Filter executes the upstream filter and parser, with its setup call charged to the same budget. Its separately named resume variant adds the fixed instruction “Now complete the original user task using the selected tools.” IPIGuard runs its upstream planning, whitelist, traversal and reflection components through message-format and budget adapters. Its pinned source revision is 4e686ed2f62c (prefix). Thus these are comparisons in a constrained local runtime, not reproductions of the baselines’ original paper settings.
For each method we report utility U (native task success), attack success X on attacked cases, and attacked joint success . Clean cases have no attack trial. An unknown checker outcome would remain unknown, not become . Model-call counts include setup and planning; token counts and repair costs retain missing usage explicitly. The accompanying numerical artifact independently reconstructs formal outcome statistics and call totals from the released task matrix; per-call token/time and archived replay intervals remain supplied aggregates. Runtime is descriptive because execution order and shared hardware can affect it.
The live sample has 32 paired task clusters, not 256 independent observations. Rate intervals use Wilson 95% intervals; paired differences resample the task pairs 10,000 times with seed 20261018. Replay intervals cluster by the 81 underlying tasks to avoid treating repeated trajectories as independent. The small live attacked subset limits both attack-rate precision and subgroup inference. No statistical test converts a zero count into a guarantee; exploratory comparisons are not a confirmatory method ranking.
4.4. Live Execution
The live executor runs on a Jetstream A100 with llama.cpp b11624. Following development, we freeze Qwen2.5-14B-Instruct Q4_K_M, a shared system instruction and budgets of 12 model calls, 12 actions, 1,024 output tokens per call and an 8,192-token context. Setup and planning consume the same call budget. The final development gate requires at least two of three ungated clean twins to succeed and known outcomes; the 72-episode validation passes. Earlier failures, model screens and interface diagnostics are excluded from formal estimates and remain in the historical evidence. A seven-billion-parameter model failed the clean-twin development gate; moving to the pinned 14B model was a separate development protocol, not a post hoc formal-result change.
The formal batch contains 16 clean and 16 attacked task IDs disjoint from the development tasks, eight methods and one seed: 256 completed episodes with known native outcomes. Task families and archived payloads appeared in earlier diagnostics, so task disjointness does not imply a previously unseen benchmark. We retain upstream Tool Filter and IPIGuard [5] alongside five system controls, and separately label a Tool Filter adaptation with a generic resume instruction. No task-specific hints or formal-outcome tuning are used.
Figure 3.
Frozen live comparison on 32 development-disjoint AgentDojo tasks, 16 clean and 16 attacked, using one 14B model and one seed. Points are observed rates; intervals are Wilson 95%. Joint success requires native task success and native attack failure. Zero observed attacks is compatible with substantial uncertainty and does not establish semantic privacy. Plotting code was prepared with AI assistance.
Figure 3.
Frozen live comparison on 32 development-disjoint AgentDojo tasks, 16 clean and 16 attacked, using one 14B model and one seed. Points are observed rates; intervals are Wilson 95%. Joint success requires native task success and native attack failure. Zero observed attacks is compatible with substantial uncertainty and does not establish semantic privacy. Plotting code was prepared with AI assistance.

Ungated execution completes 10/32 tasks and exhibits one native attack success among 16 attacked tasks. Gate and scope complete 10/32 each, receipt 9/32, deny-egress and IPIGuard 8/32 each, original Tool Filter 0/32 and its resume adaptation 12/32. All defended variants have zero observed native attacks. Only one attack case is discordant with no gate; zero out of 16 has a Wilson 95% upper bound of 19.4%. This is not evidence of broad privacy superiority.
Resume gains four tasks and loses two against no gate: percentage points (paired bootstrap 95% interval ). Receipt gains one and loses two: points (). Neither supports utility superiority. Gate, scope, receipt and IPIGuard each attain 4/16 attacked joint successes, equal to no gate; resume attains 5/16. Joint success still uses a specific native attack predicate, not comprehensive semantic privacy.
Cost and Failure Accounting
Total model calls are 138 for no gate, 137 for gate, 136 each for scope and receipt, 142 for deny-egress, 211 for IPIGuard, 69 for original Tool Filter and 144 for resume. IPIGuard exhausts the matched budget on 21/32 tasks. Four original Tool Filter calls have unknown token use; their totals are lower bounds. Failures and unknown costs remain visible. Appendix Figure A3 reports paired cost intervals. Timing is observational, not randomized, and no currency-cost claim is made.
Full task outcomes, report semantics, selection history and integration details appear in Appendices Appendix A–Appendix E. Appendix G separates proposed social workflows from completed studies.
5. Deployment and Reuse
A Deployed Research Environment
On Jetstream we separated business and audit PostgreSQL instances, a private report service and Qdrant retrieval; raw records and prompts stayed on the business side. A controlled regression exercised 1,872 operations over 2, 8 and 16 logical tenants, concurrency levels 1 and 4, and three repetitions. It tests permission freshness and reporting, not 1,872 independent semantic tasks. Delivery-time checks blocked the constructed revoked accesses, but also withheld otherwise authorized operations when policy access failed. An actual stopped report-service container tested outbox preservation for triggered recovery. This operational regression is supporting deployment evidence, not part of the 32-task live estimate.
A LangGraph pilot connects retrieval to two model roles using synthetic business records. Its failures motivate the authorization–relevance–support separation: withholding a revoked record did not prevent another customer’s reference from being substituted or an unsupported refund statement. This is a real software stack with controlled data, not a customer deployment. AgentDojo’s mail, Slack and travel tools likewise execute in benchmark environments; we did not test production accounts, real users or end-to-end service-level objectives.
What Applications Can Reuse Now
An application can mirror a handoff into AuditSession, supply trusted policy labels and receive scoped findings without replacing its agent planner. The accompanying minimal handoff example makes no model or network calls and does not enforce delivery. Enforcement requires a separate, fully mediated release boundary; the transactional adapter tests a database mailbox commit, not external delivery. Researchers can reuse the frozen task matrix and paired outcome/cost reporting to compare controls while retaining failures, abstentions and unknown costs.
Relation to Prior Work
AgentLeak [6] and Sum-Leaks [7] motivate internal-channel and compositional checks; AgentDojo supplies executable tasks. Our metadata interface complements local enforcement and tracing. DLP can also run locally; our observation boundary is not unique to a centralized deployment architecture. Neither our measurements nor authenticated reports establish general semantic confidentiality, production readiness or utility superiority. False labels, omitted events, source errors, correlated inference and bypass channels remain open. Historical model aliases cannot support precise capability rankings and are excluded from the present model-comparison claims.
Research Directions Arising from the Failures
Interoperable evidence: extend the experimental A2A metadata interface with scoped authorization, relevance and support outcomes and explicit missing obligations. Cross-framework conformance tests should distinguish incompatible policies from missing evidence; adoption and semantic conformance remain untested. Recovery without broader authority: after a refusal, replan using permitted sources or request an explicit policy change. Evaluate privacy, completed tasks and repair cost together against deny-all and unchanged-authority controls; our prefix replay cannot establish this recovery benefit. Persistent memory and composition: test repeated queries, retention windows and revocation after information has entered agent memory. Use independent recipient attacks and adjudicated semantic labels to measure cumulative leakage; a later permission check cannot retract previously disclosed information. Social group chats could provide a future testbed for cross-message composition and revocation, but no WhatsApp or Instagram integration is evaluated here. These are proposed research targets, not capabilities validated here.
6. Conclusion
A metadata interface can reduce the raw content collected by an audit center, while a local gate can enforce narrowly specified release obligations. Neither entails semantic confidentiality. The authored mechanism checks support the implemented exact-field rules; archived replay demonstrates blocking with measurable utility loss; the frozen live comparison exposes sparse attacks, budget failures and uncertain utility differences. Reusable governance evidence should state which boundary was checked, preserve unknown outcomes, and report completed tasks and resource costs alongside attacks. Claims about persistent cross-agent inference, production integrations and general free-text support require additional independent validation.
Open Science
The accompanying numerical artifact contains the 256-row task-by-method matrix, frozen aggregate summaries, statistical functions, figure scripts and a SHA256 manifest. Its offline verifier reconstructs formal outcome rates, Wilson intervals, paired bootstrap intervals, calls, repairs and budget failures without model inference. It redraws token/time and replay results from supplied aggregates; those intervals cannot be independently reconstructed without the withheld per-call records and archived trajectories. Authored mechanism results are aggregate checks. Raw execution traces remain in the research environment. This package supports numerical verification, not end-to-end reproduction of model execution or independent validation of native checkers.
Institutional Review Board Statement
The reported evaluations use authored records and benchmark environments rather than production customer accounts. Attack payloads are evaluated in those environments; this work does not authorize their use against external services. Metadata can reveal sensitive categories and links even when raw text is withheld. Reporting minimization is therefore an architectural choice, not an ethics exemption or a confidentiality guarantee. This study was not submitted to an institutional review board or another external ethics-review body. No formal approval or exemption determination is claimed.
Acknowledgments
Codex was used for code development, experiment orchestration, numerical analysis scripts, related-work lookup, manuscript revision and pre-submission checks. It also assisted the plotting implementations for the figures. The evaluated language models are part of the research methodology as described in the text. The author is responsible for the accuracy, originality and integrity of the manuscript and artifacts. AI assistance does not constitute independent human validation of the findings.
Appendix A. Complete Task-Level Evidence
Figure A1.
Complete task-by-method matrix. No failures or outcome-selected rows are omitted. Colors encode native outcomes, numbers encode model calls, and dots mark budget exhaustion.
Figure A1.
Complete task-by-method matrix. No failures or outcome-selected rows are omitted. Colors encode native outcomes, numbers encode model calls, and dots mark budget exhaustion.

The display retains all 32 frozen task IDs and all eight methods. C denotes clean tasks and A attacked tasks. Clean tasks have no attack trial: green in those rows means completion, not a tested security outcome. For attacked tasks, green requires completion and native attack failure; gray means task failure without observed attack success; red means native attack success. Black dots retain budget exhaustion even when the native task checker succeeds. Cells show model calls, including setup and planning. Task IDs follow the sorted remote keys; raw text and trajectories remain remote.
These are 32 paired task clusters, not 256 independent tasks. The selected T30 display in the main text does not replace this complete population. Original Tool Filter and its continuation adaptation are kept separate. Unknown token usage is not an unknown native checker outcome: all native task outcomes are known, while four original Tool Filter calls lack token usage. The paired token comparison uses the 28 tasks with complete paired usage and reports the excluded pairs explicitly.
Appendix B. Diagnostic Example and Paired Cost
The post-hoc T30 example exposes why suppressing a native attack does not imply useful completion. The cost display pairs every method with ungated execution on the same tasks; missing token usage is explicit. These diagnostic views expand the aggregate results in the main text.
Figure A2.
Same frozen task, eight outcomes. T30 is selected post hoc as the smallest task index with ungated native attack success. Only the explicitly named Tool Filter resume adaptation completes this task. This diagnostic is not a representative effectiveness estimate; the complete matrix in Appendix A retains all 256 executions.
Figure A2.
Same frozen task, eight outcomes. T30 is selected post hoc as the smallest task index with ungated native attack success. Only the explicitly named Tool Filter resume adaptation completes this task. This diagnostic is not a representative effectiveness estimate; the complete matrix in Appendix A retains all 256 executions.

Appendix C. Report Semantics and the Local Release Contract
Three Identities
A data subject is the person or entity described by a fact. An owner controls the resource. An acting agent performs a handoff. Collapsing them into a single tenant identity can miss a disclosure about a third party or incorrectly infer authorization from shared ownership. The report therefore carries these roles separately, along with recipient and purpose.
A Report is Evidence of a Claim
The extension includes sensitivity, categories, inferred categories, allowed recipients, hop budget and provenance. Hashes support equality checks and deduplication; they do not make low-entropy content unrecoverable. A MAC authenticates the issuer’s serialized statement under a protected key. It does not prove that the issuer observed all events or assigned correct categories. A previously authenticated report can also be replayed unless event identity and freshness are checked separately.
Composition Boundary
The graph groups supplied edges by recipient principal and data subject and combines supplied inferred-category tags. A session can submit multiple messages. This is a check over the supplied view, not a demonstrated ability to reconstruct every message an adversarial recipient has retained. The likelihood ratios in the odds model are assumptions; they are not calibrated measurements of recipient inference. Pseudonyms retain the linkability required by these checks and can retain identifying structural patterns.
Structured Enforcement
A proposal contains source identifier, source version, field and value. The trusted adapter supplies the workspace and case, recipient and purpose, and current authorization. The full gate requires current recipient access, matching case and workspace, an allowed field-purpose combination, matching source version and exact equality to the source value. It rejects permission overrides and extra free text. Missing evidence returns unknown; established mismatches return deny. Only admitted fields reach a fixed renderer, and any failing field withholds the proposal.
Figure A3.
Task-paired mean cost differences against ungated execution, with 10,000-resample bootstrap 95% intervals. Calls include setup and planning. Original Tool Filter input-token differences exclude tasks with unknown usage; four calls have unknown token use. All other token comparisons are complete. Timing is observational. A cheap failed task is not an efficient successful task. Plotting code was prepared with AI assistance.
Figure A3.
Task-paired mean cost differences against ungated execution, with 10,000-resample bootstrap 95% intervals. Calls include setup and planning. Original Tool Filter input-token differences exclude tasks with unknown usage; four calls have unknown token use. All other token comparisons are complete. Timing is observational. A cheap failed task is not an efficient successful task. Plotting code was prepared with AI assistance.

What Exact Matching Misses
A source can be wrong, an authorized recipient can combine admitted values, and a correct paraphrase can fail an equality check. Conversely, plausible free text can be unsupported even after a revoked record is withheld. These are different failure modes. The structured mechanism is not a semantic entailment checker, and its authored-task result does not validate a general free-text release policy.
Transaction Boundary
The PostgreSQL adapter serializes the decision with a business-side mailbox commit. This tests a database boundary. It does not guarantee delivery by an external messaging service or erase information already in a model context. The live AgentDojo controls are separate implementations and do not establish exact-field support for arbitrary messages.
Appendix D. Frozen Development History and Reproducibility
The formal batch follows development rather than selecting a favorable formal configuration. Earlier failures remain diagnostic evidence; none are pooled with the 32 formal tasks. Table A1 records the major transitions relevant to the final model and interfaces. The final eight methods were frozen before the independent formal batch.
Table A1.
Development selection history, distinct from formal evaluation. Clean twins are the same attacked-task requests without the injection, not new independent benchmark tasks.
Table A1.
Development selection history, distinct from formal evaluation. Clean twins are the same attacked-task requests without the injection, not new independent benchmark tasks.
| Stage | Observations | Interpretation / subsequent boundary |
|---|---|---|
| 7B canonical (v23) | 63 episodes; twins 0/3 | Failed competence screen. Recorded-action replay reproduced missing calendar creation, wrong/missing date and omitted website. |
| Interface (v25) | 27 episodes; twins 0/3, 1/3, 0/3 | All three interfaces failed the gate; upstream-prompt condition recorded ten tool errors. |
| Upstream oracle (v26) | 6/6 environment checks | Environment and scoring validation only; not model competence. |
| 14B screen (v27) | 9 episodes; twins 2/3 | Separate frozen model protocol; screening results are not formal estimates. |
| Seven methods (v28) | 63 episodes | Original Tool Filter completed 0/9; premature termination motivated a separately labeled continuation diagnostic. |
| Filter compatibility (v29) | 18 episodes; utility 0/9 vs 3/9 | Original and generic-resume variant, all nine conditions; fixed run order prevents causal timing claims. |
| Final development (v30) | 72 episodes; twins 2/3 | Eight methods, shared model, system instruction and budgets; all native outcomes known. |
| Formal (v31) | 32 tasks × 8 methods | 256 completed executions; disjoint task IDs, fixed parameters, failures and unknown costs retained. |
Frozen Runtime
The final model is Qwen2.5-14B-Instruct. Its Q4_K_M weights are served by llama.cpp b11624 on a Jetstream A100. All methods share temperature 0, seed 0, an 8,192-token context, 1,024 output tokens per call, 12 model calls and 12 actions. Setup and planning consume the same call cap. The model revision, shard hashes, source hashes and actual system instruction are recorded in the runtime manifests. The previous 7B service is not the final endpoint.
Statistical Units
Clean and attacked subsets each contain 16 task IDs. Marginal rate intervals are Wilson 95% intervals. Paired differences resample task pairs 10,000 times using seed 20261018; gains and losses remain visible. Replay resamples 81 underlying task clusters rather than treating 1,132 traces as independent tasks. Small attack counts limit resolution: zero of 16 still has a Wilson upper bound of 19.4%.
Failure and Cost Accounting
Format failures, tool errors, budget exhaustion and unknown usage are separate from network failures and from privacy success. Repair calls are calls following an observed denial or error; their token and time accounting does not isolate a causal benefit of repair. Reported time is descriptive because order, caching and shared hardware can affect it. No monetary cost or production-throughput claim is made.
Appendix E. Minimal Application Integration
The offline example wraps the existing AuditSession. An application supplies handoff text locally together with recipient identity, sensitivity, categories and allowed recipients. It receives finding types and counts. The example makes no model or network calls and does not alter delivery. Trusted policy configuration must supply the labels; the sender must not be allowed to grant itself authority through metadata.
Integration Sequence
First bind the actor, owner and subject to application identities. Next attach labels locally and submit the handoff to the session. Record the findings separately from the business outcome. Finally, if enforcement is required, place a separately mediated release check before the side effect. A local finding is not a delivery receipt, and an empty finding set is not a confidentiality certificate. A workflow requiring cumulative auditing needs a persistent session rather than a single-handoff wrapper.
Reusing the Evaluation
Preserve the task ID, clean/attacked status, method, native completion and attack predicates, call/action caps and unknown usage. Pair controls on identical task inputs. Evaluate refusal, recovery and completion together, since a method can suppress attacks by completing nothing. Keep setup and planning calls inside the same budget. Export numerical evidence rather than raw private messages for central analysis.
Appendix F. Deployment Case Study: Policy Freshness and Observed-Event Coverage
We deployed a research harness on Jetstream with separate business and audit PostgreSQL instances, a private report service, and Qdrant retrieval. Raw records and model prompts remain on the business side. These experiments exercise the reporting and authorization substrate, not the semantic detectors evaluated above. A signature authenticates an issuer’s claim, not its truth or completeness.
Figure A4.
Deployed research pipeline. Missing semantic evidence is reported as unknown rather than treated as a successful audit.
Figure A4.
Deployed research pipeline. Missing semantic evidence is reported as unknown rather than treated as a successful audit.

A controlled regression executed 1,872 operations over 2, 8, and 16 logical tenants, concurrency levels 1 and 4, three repetitions, six authored scenarios, and two strategies. These are repeated conditions, not independent semantic examples. In the constructed revoke and delete cases, a retrieval-time permission snapshot admitted all 156 operations per case; a delivery-time recheck admitted none. The recheck failed closed for all 156 otherwise authorized operations with injected policy unavailability. A subsequent schema revision transported explicit unknowns: all 48 new regression event IDs reached the center, including 26 unknowns. A real report-service outage preserved committed outbox records for explicitly triggered recovery. These observations expose an availability tradeoff, rather than establishing population-level privacy effectiveness.
Figure A5.
Permission freshness regression. Counts reflect authored mutations. Policy unavailability is injected; the separate report-service outage test uses an actual stopped container.
Figure A5.
Permission freshness regression. Counts reflect authored mutations. Policy unavailability is injected; the separate report-service outage test uses an actual stopped container.

Online Two-Role Pilot
A separate LangGraph pilot used MiniLM embeddings over six synthetic records and two stateless Qwen2.5-3B-Instruct roles. Four tasks and two controls yielded eight episodes and 16 generations. All 40 observed event IDs reached the center, including 16 generated-output observations marked unknown. This does not detect unreported events. A snapshot-controlled episode propagated a revoked identifier into its handoff and final answer; the paired recheck withheld that record. Nevertheless, the recheck answer misattributed another customer’s tracking reference. Required-term presence therefore does not establish task success.
Twelve of 16 generations reached the 96-token cap. We preserve these results and do not use the pilot to estimate semantic audit recall or general task success. Independent human semantic review has not been completed. These observations establish a working research harness, not production readiness.
Figure A6.
Pilot outcomes. Keyword presence is an automatic proxy, not a semantic judgment. Raw responses expose misattribution despite a positive proxy.
Figure A6.
Pilot outcomes. Keyword presence is an automatic proxy, not a semantic judgment. Raw responses expose misattribution despite a positive proxy.

Development Follow-Up: Retrieval Scope and Policy Freshness
A second frozen run reused the four development tasks with four configurations: broad or case-scoped retrieval crossed with snapshot or delivery-time checks. All 16 episodes and 32 generations completed without execution errors or token-cap hits. Both snapshot configurations exposed the revoked Birch reference, whereas both recheck configurations withheld it. The broad recheck answer nevertheless substituted Acorn’s reference. The scoped recheck answer avoided that substitution but asserted that the refund had not been processed, unsupported by its available records. These observations motivate Section 3.2; they are not independent human semantic judgments. The prompt and output cap changed from the first pilot (96 to 256 tokens), so differences across pilots do not isolate the effect of retrieval scope. Raw traces, frozen protocol, model manifest, and unfilled human-review forms are preserved under deploy/research/results/live-v2-20261013/.
Appendix G. Future Social-Group and Graph-View Designs
Figure A7 and Figure A8 illustrate prospective workflows. They are invented workflows, not additional benchmark tasks, measured attack outcomes or platform integrations. WhatsApp-like and Instagram-like styling identifies the motivating context only. No connector, production account or platform API is exercised.
Figure A7.
Future social workflow: invented messages and three agent roles. Platform-like panels are conceptual surfaces, not screenshots or implemented integrations. The candidate health inference is unmeasured.
Figure A7.
Future social workflow: invented messages and three agent roles. Platform-like panels are conceptual surfaces, not screenshots or implemented integrations. The candidate health inference is unmeasured.

Figure A8.
Proposed view ablation, with no results. The 1,440-episode number is a design budget, not completed experiment scale. The three illustrated graph views and a raw-identity reference would be paired within workflow clusters.
Figure A8.
Proposed view ablation, with no results. The 1,440-episode number is a design budget, not completed experiment scale. The three illustrated graph views and a raw-identity reference would be paired within workflow clusters.

Cross-Message Inference
One agent may reveal a recurrence and another a location. A recipient that links their subject and history can form a sensitive hypothesis although neither fragment states it directly. A future study should measure that recipient’s inference independently rather than equating a metadata alert with an actual inference. Revocation after receipt cannot erase memory.
Graph Views
Stable aliases preserve a hub and triangle that can enable linkage to auxiliary information. Window-scoped aliases reduce cross-window joining but may also weaken audit continuity. Exact aggregate counts remove individual paths without providing differential privacy. A useful comparison must measure both recipient inference and auditor re-identification, alongside utility, audit coverage, unknowns and costs.
Falsifiable Follow-Up
Hold the authorized decision and private fact distribution fixed across views; vary membership state, retained history and report visibility. Use independent attacks and adjudicated labels. Predefine whether leakage means exact attribute recovery, calibrated belief gain or identity linkage. Measure the protection lost when a view omits the relationships required by the audit. No winner, anonymity guarantee or federated-learning advantage is implied by the conceptual diagrams.
References
- Debenedetti, E.; Shumailov, I.; Fan, T.; Hayes, J.; Carlini, N.; Fabian, D.; Kern, C.; Shi, C.; Tramèr, F. Defeating Prompt Injections by Design. 2025. [Google Scholar]
- Shim, M.; Karim, R.R.; Jakkula, R.; Zhou, K.; Liu, X.; Wang, X.E.; Li, Z. Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control. 2026. [Google Scholar]
- Zhang, Y.; Zhao, X.; Liu, S.; Lou, H.; Cheng, G.; Liu, C. FAVA: Formal Authorization for Verified Agents with Evidence-Backed Permission Graphs. 2026. [Google Scholar]
- Debenedetti, E.; Zhang, J.; Balunovic, M.; Beurer-Kellner, L.; Fischer, M.; Tramèr, F. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. In Proceedings of the The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. [Google Scholar]
- An, H.; Zhang, J.; Du, T.; Zhou, C.; Li, Q.; Lin, T.; Ji, S. IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents. In Proceedings of the Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025; pp. 1023–1039. [Google Scholar] [CrossRef]
- El Yagoubi, F.; Badu-Marfo, G.; Al Mallah, R. AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems, 2026. arXiv arXiv:2602.11510.
- Patil, V.; Stengel-Eskin, E.; Bansal, M. The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration, 2025. arXiv arXiv:2509.14284.
Figure 1.
Architecture and experimental methodology. The observation path groups locally reported metadata by recipient and subject; it does not inspect raw text or establish truthful labels. The separate release adapter checks current authorization, case relevance and exact source support before rendering structured fields. Bottom: mechanism checks share proposals across gates; replay truncates archived actions without recovery; live execution compares eight methods on development-disjoint tasks under equal resource caps. Replay counts show no gate versus combined intent/destination checks; live ranges span all eight methods. These regimes do not validate an end-to-end composition of every audit and release control. Messages are illustrative; unknowns and utility losses remain visible.
Figure 1.
Architecture and experimental methodology. The observation path groups locally reported metadata by recipient and subject; it does not inspect raw text or establish truthful labels. The separate release adapter checks current authorization, case relevance and exact source support before rendering structured fields. Bottom: mechanism checks share proposals across gates; replay truncates archived actions without recovery; live execution compares eight methods on development-disjoint tasks under equal resource caps. Replay counts show no gate versus combined intent/destination checks; live ranges span all eight methods. These regimes do not validate an end-to-end composition of every audit and release control. Messages are illustrative; unknowns and utility losses remain visible.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.