RAGSec: A Defense Framework for RAG Under Adaptive Knowledge Poisoning

1. Introduction

Retrieval-augmented generation (RAG) attaches an external document collection to a language model so that answers are grounded in text retrieved at inference time rather than in model parameters alone [1,2,3]. The appeal for enterprise and government deployments is operational: the knowledge base can be corrected, extended and audited without retraining the generator, and answers can be traced to sources. That same separation, however, moves part of the trust boundary out of the model and into a data pipeline. An adversary who can influence what enters the index, or what is retrieved for a given query, can influence the model’s output without ever touching its weights.

This is not a hypothetical concern. Retrieval corpora are routinely assembled from crawled web pages, ticketing systems, wikis, shared drives and third-party connectors, and web-scale collections have been shown to be poisonable at low cost [4]. A growing body of work demonstrates concrete attacks: passages crafted to be retrieved for a chosen question and to carry a false answer [5,6], dormant passages that activate only when a rare trigger appears in the query [7,8,9], instructions embedded in retrievable content that hijack the generator’s instruction hierarchy [10,11], and volumes of topically matching but uninformative text that deny service by crowding genuine evidence out of the context window [12]. The defensive literature has responded with retrieval-time detectors, robust aggregation over retrieved passages, and certified schemes [13,14,15].

Two problems recur in how these defenses are evaluated. First, they are usually measured against the published, static form of each attack, that is, the exact string construction the original paper proposed. A heuristic detector that keys on that construction will report near-perfect accuracy, which says little about what happens when an adversary who has read the defense simply writes the payload differently. The adversarial machine learning literature has made this argument repeatedly and at length: defenses that are not evaluated against an adaptive attacker routinely collapse when one appears [16,17,18,19]. Second, the layers of a defense stack are rarely separated, so it is hard to tell which control is carrying the security and which is decorative.

This paper addresses both. We implement ragsec, an instrumented RAG pipeline in which ingestion verification, retrieval filtering, robust aggregation and anomaly monitoring are independent, individually switchable layers, and we evaluate it against five attacks: the four static families above plus an adaptive adversary that we construct specifically to defeat the deployed stack. The adaptive adversary is parameterized by six independent evasion techniques, so we can attribute its success to particular techniques and, symmetrically, identify which defensive assumption each layer silently depends on.

The headline result is negative for content-based defense and positive for provenance-based defense. Against the static attacks the layered architecture behaves as the literature would predict: attack success falls from near unity to approximately zero, at a small utility cost. Against the adaptive adversary the same stack fails almost completely: the retrieval filter’s true-positive rate falls to near zero, and attack success remains close to that of the undefended pipeline, and on one corpus far above it. Binding ingestion to an unforgeable signature, rather than to a claimed source name and trust score, restores the defense. A leave-one-out ablation shows the adaptive attack needs exactly three of its six techniques, and that the two content-level detectors it must evade are each defeated by a single, cheap rewriting choice.

Our main contributions are the following:

• A provenance-aware RAG pipeline with four independently switchable defense layers and a common hook interface, enabling clean per-layer ablation under a fixed protocol (Section 4);
• A defense-aware adaptive poisoning attack with six independently controllable evasion techniques, which allows the causes of defensive failure to be attributed rather than merely observed (Section 5);
• A calibrated suspicion score that places four heterogeneous retrieval filters on a common scale, making the detector’s operating point and its ROC well defined (Section 4.2);
• An evaluation protocol that separates attack-attributable failures from the pipeline’s own error rate, which matters for suppression attacks whose naive success rate is inflated by baseline errors (Section 6.3); and
• An experimental study over three corpora, seven configurations, five attacks and three seeds, reporting where the layered architecture holds, where it fails, and what each layer costs in utility and latency (Section 7).

The remainder of this paper is organized as follows. Section 2 reviews related work, and Section 3 defines the threat model. Section 4 presents the four-layer defense architecture, while Section 5 describes the defense-aware adaptive adversary. Section 6 details the experimental design, and Section 7 reports the results and ablation studies. Section 8 discusses the implications for RAG security, Section 9 outlines the limitations, and Section 10 concludes the paper.

2. Related Work

2.1. Retrieval-Augmented Generation

RAG couples a dense retriever with a generator, so that a query is answered from passages selected at inference time [1,2]. Dense retrieval with dual encoders [20,21] and approximate nearest-neighbor indexes [22] made the approach practical at scale, and subsequent architectures refined how retrieved evidence is fused and when it is used [23,24,25,26]. Surveys document its rapid adoption [3]. A parallel line of work shows that RAG pipelines are sensitive to what they retrieve even absent an adversary: irrelevant or distracting passages measurably degrade answer quality [27,28], which is the benign analog of the availability attack we study.

2.2. Poisoning the Knowledge Base

Corpus poisoning predates RAG. Classical data poisoning manipulates a training set [29], and backdoor attacks plant input-conditioned behavior [30]; Carlini et al. [4] showed that poisoning web-scale collections is practical and cheap. In the retrieval setting, Zhong et al. [6] demonstrated that a small number of adversarial passages inserted into a corpus can be retrieved for a broad range of queries. PoisonedRAG [5] formalizes targeted knowledge corruption as a joint retrieval and generation condition: a poisoned passage must be retrieved for the target query and must, once retrieved, induce the adversary’s answer; the canonical construction concatenates the target query with an adversarial statement so that the passage embeds close to the query. Trigger-conditioned variants keep the poison dormant for ordinary traffic and activate it on a rare phrase [7,8,9]. Availability-oriented poisoning replaces answer substitution with answer suppression [12], and Tan et al. [31] study how retriever weaknesses themselves can be exploited. Deng et al. [32] apply corpus poisoning to elicit unsafe generations.

2.3. Indirect Prompt Injection

Where poisoning targets what the model believes, prompt injection targets what it obeys. Perez and Ribeiro [33] showed that instruction-shaped text in the input can override a system prompt, and Greshake et al. [10] showed that in an LLM-integrated application this text need not come from the user at all: it can arrive through retrieved documents, email or web pages. Liu et al. [34] and Liu et al. [11] systematize the attack and the available defenses. Proposed mitigations operate at different levels: training the model to respect an explicit instruction hierarchy [35], structured prompting that keeps data out of the instruction channel [36,37], and marking retrieved content so the model can distinguish it from instructions [38]. Detection-based defenses filter suspicious inputs, for example by perplexity [39,40]; our Layer 2 is in this family and inherits its weakness against paraphrase.

2.4. Defending the Retrieval Pipeline

RobustRAG [13] introduced isolate-then-aggregate: each retrieved passage is used to produce an answer independently and the answers are then aggregated, so that a minority of corrupted passages cannot dominate the generation; the scheme admits certifiable robustness guarantees under a bound on the number of corrupted passages. TrustRAG adds clustering and consistency checks over retrieved evidence [14], and Weller et al. [15] defend open-domain QA against disinformation by exploiting redundancy and query augmentation. Our Layer 3 is a provenance-weighted variant of isolate-then-aggregate; our results indicate that the corrupted-minority assumption underlying this family is precisely what a corroborating adaptive adversary attacks.

2.5. Provenance and Supply-Chain Integrity

Outside machine learning, the response to untrusted artifacts entering a pipeline has converged on cryptographically verifiable provenance rather than content inspection: signed artifacts and transparency logs [41], attested supply chains [42], and secure development practice [43]; the underlying primitive is standard message authentication [44]. The motivating threat, malicious packages entering a trusted-looking distribution channel [45], is structurally the same as the one we study, in which an adversary uploads to a connector the RAG deployment already trusts. Contemporary risk taxonomies for AI systems identify both data poisoning and prompt injection as primary risks [46,47]. Our findings support importing the supply-chain answer wholesale: the only control in our evaluation that survives an adaptive adversary is signature-bound ingestion.

2.6. Evaluating Defenses Against Adaptive Adversaries

A defense evaluated only against the attack it was designed for measures its own training set. This lesson was learned expensively in adversarial examples, where a long sequence of detection defenses was broken by attacks tailored to them [16,17,18], leading to explicit methodological guidance [19]. To our knowledge, defenses for RAG poisoning have not been subjected to the same discipline: reported evaluations are dominated by static attack constructions. Section 5 supplies such an adversary, and Section 7 reports what happens.

2.7. Comparison with Prior Work

Table 1 summarizes the approaches discussed above by their security focus, mechanism, and relationship to RAGSec. The comparison is qualitative: the studies address different threat models and are not evaluated here as interchangeable baselines. Attack-oriented work motivates our poisoning scenarios, while instruction separation, retrieval filtering, and evidence aggregation address different points in the pipeline. RAGSec brings these control points into an instrumented framework to examine which layers remain effective when an adversary adapts to their decision rules.

The distinction is therefore primarily methodological rather than a claim of a new cryptographic primitive or a stronger certified aggregation rule. Our experiments separate the contribution of each defense layer, identify which adaptive rewriting choices defeat the heuristic stack, and compare content-based controls with authenticated ingestion. These conclusions remain bounded by the evaluated corpora, attack constructions, and generator implementations; the comparison does not establish superiority over the original systems on their respective benchmarks.

3. Threat Model

3.1. System Model and Notation

Let $\mathcal{D}=\{d_1,\dots ,d_n\}$ be the document collection. Each document d carries provenance $\pi \left(d\right)=(s,\sigma ,\omega )$, where s is the identifier of the source that submitted it, $\sigma$ an optional signature over its content, and $\omega \in [0,1]$ a trust score associated with s. Documents are split into chunks and embedded by $E:{\mathsf{\Sigma }}^{*}\to {\mathbb{R}}^h$ with unit-norm output, so that the inner product $\langle E\left(q\right),E\left(c\right)\rangle$ is the cosine similarity between a query q and a chunk c. The retriever returns the $mk$ highest-scoring chunks, where k is the number of passages the generator ultimately sees and $m\ge 1$ is an over-retrieval (candidate) multiplier that exists so that chunks removed by a defense can be replaced by clean evidence rather than shrinking the context. Write $R\left(q\right)$ for this candidate set. A generator G maps a query and a passage set to an answer $a=G(q,P)$.

The undefended pipeline is $a=G\left(q,{\mathrm{top}}_{k}R\left(q\right)\right)$. A defended pipeline interposes four functions: $f_1$ filters documents at ingestion, $f_2$ filters candidates at retrieval, $f_3$ replaces the single generator call with an aggregation over isolated calls, and $f_4$ observes the resulting stream without modifying it.

3.2. Adversary Model

We assume an adversary who can write to the knowledge base, whether by uploading through a public or semi-public connector, by compromising a single ingestion account, or as a malicious insider, and who knows the target question. Following standard practice for security evaluation, we grant the adversary knowledge of the defense: its architecture, the filters it deploys, and its thresholds. We do not grant the adversary the ability to modify the generator’s weights, to read or forge the ingestion signing key, to modify the retriever, or to observe other users’ queries. The adversary’s budget is the number of documents b they may insert per target question; we sweep b explicitly.

Two adversarial goals are distinguished. Under substitution, the adversary chooses a target answer $a^{*}$ and succeeds when $a^{*}$ appears in the system’s response. Under suppression, the adversary succeeds when the correct answer does not appear; this covers denial of service against a knowledge base.

3.3. Attack Taxonomy

Table 2 summarizes the five attacks. A1–A4 are static: each reproduces a published construction and makes no attempt to evade detection. A5 is the adaptive adversary introduced in Section 5.

4. Defense Architecture

Figure 1 summarizes the document-to-answer pipeline, the attack entry point, and the evaluation workflow. Each layer implements a common interface with four optional hooks (on_ingest, on_retrieve, generate and on_response), so any subset of layers can be enabled without altering the retriever, the generator or the evaluation protocol. Section 4.1, Section 4.2, Section 4.3 and Section 4.4 describe the layers in turn. The design follows three principles: place controls at distinct trust boundaries rather than stacking detectors at one; make every layer independently measurable; and prefer conservative failure (abstention) to a possibly manipulated answer.

4.1. Layer 1: Source Verification

Layer 1 decides what may enter the index. It applies, in order: a source allowlist; a minimum trust threshold ${\theta }_{\mathrm{trust}}$ on the trust score associated with the claimed source; HMAC-SHA256 signature verification [44] recomputed over the document text rather than trusted from caller-supplied metadata; content-hash deduplication; and a per-source document quota that bounds bulk insertion. Requiring signatures without a configured verification key is rejected at construction time rather than silently degrading to a no-op.

Algorithm 1 gives the order of checks. The distinction that turns out to matter experimentally is between the claimed and the provable parts of provenance. A source identifier and a trust score are attributes an uploader asserts; a signature is one they must be able to produce. Allowlists and trust thresholds constrain only the first kind.

Algorithm 1 Layer 1: ingestion-time source verification

4.2. Layer 2: Calibrated Retrieval Filtering

Layer 2 scores every candidate in $R\left(q\right)$ with four detectors that target disjoint mechanisms. The instruction-pattern filter applies NFKC normalization and strips invisible characters before matching override directives and chat-template markers, so that zero-width obfuscation does not evade it. The query-echo filter measures bigram overlap between the query and the passage and separately detects passages that open with a question, the construction shared by A1, A3 and A4. The similarity-outlier filter flags candidates that are near-duplicates of the query in embedding space, and candidates whose distance from the corpus centroid is an outlier by more than $z_{max}$ standard deviations. The naturalness filter measures gibberish token ratio, token repetition, invisible characters and character entropy.

These detectors return incomparable statistics: a count, a ratio, a cosine, a z-score. To give the layer a single, sweepable operating point we normalize each by its own decision threshold, so that a calibrated severity of 1 reproduces exactly that detector’s own verdict:

$${\mathrm{sev}}_i(c,q)=\mathrm{clip}\left({\displaystyle \frac{{\mathrm{stat}}_i(c,q)}{{\theta }_i}},0,1.5\right),\mathrm{susp}(c,q)=\underset{i}{max}{\mathrm{sev}}_i(c,q).$$

(1)

We aggregate by maximum rather than by sum or mean because the four detectors address disjoint attack mechanisms: one confident detector should not be diluted by three that have nothing to say about the passage in question. A candidate is dropped when $\mathrm{susp}(c,q)\ge \tau$. Sweeping $\tau$ traces the detector’s ROC curve, which we report in Section 7.7; $\tau =1$ recovers the layer’s default behavior.

Algorithm 2 Layer 2: calibrated retrieval filtering

4.3. Layer 3: Provenance-Weighted Robust Aggregation

Layer 3 replaces the single stuffed-context generation with isolate-then-aggregate in the manner of RobustRAG [13], adding provenance weighting. Each surviving passage is used alone to produce a candidate answer; abstentions are discarded. Candidate answers are grouped by Jaccard overlap of their content tokens, and each group is scored by

$$W\left(g\right)=\left({\sum }_{j\in g}{\omega }_j·max{({\rho }_j,{\rho }_{min})}^{\gamma }·{\varphi }_j\right)·\left(1+\beta \left(\left|\mathrm{docs}\right(g\left)\right|-1\right)\right),$$

(2)

where ${\omega }_j$ is the source trust of the passage supporting answer j, ${\rho }_j$ the fraction of addressable query terms the answer covers, $\gamma =2$ sharpens the preference for on-topic answers, ${\varphi }_j$ is a penalty applied when Layer 2 flagged the passage, and $\beta$ is a redundancy bonus rewarding groups supported by several distinct documents. The layer returns the highest-weighted group’s representative answer only if that group both clears a minimum weight and exceeds the runner-up by a margin $\mu$; otherwise it abstains.

The redundancy bonus encodes the assumption that independent corroboration is evidence of truth. Section 7.5 examines what happens when the adversary supplies the corroboration.

Algorithm 3 Layer 3: isolate-then-aggregate with provenance weighting

4.4. Layer 4: Anomaly Monitoring

Layer 4 is detective, not preventive: it emits structured alerts and never rewrites an answer or removes a document. It maintains a sliding window of retrieved chunk identifiers and reports when the Jensen–Shannon divergence [48] between consecutive windows exceeds a threshold, which is the signature of a corpus whose retrieval distribution has been displaced; it re-runs registered benchmark queries and reports ranking shifts; and it executes canary queries with known expected answers. Including it as a separate layer lets us state explicitly what detection does and does not contribute, a point the ablation makes concrete.

Algorithm 4 Layer 4: retrieval-drift monitoring

4.5. Composition

Algorithm 5 shows how a defended query is served. Note the ordering constraint: over-retrieval must happen before filtering, or dropping a candidate simply shortens the context and hands the adversary a partial suppression win for free. Section 7.9 quantifies this.

Algorithm 5 Defended query path

5. An Adaptive Adversary

The static attacks A1–A4 are, from a detector’s point of view, easy targets: each has a syntactic tell. A1 and A4 begin with the target question; A3 contains an imperative override; A2 contains a low-entropy nonsense trigger. A defense-aware adversary has no reason to retain any of these properties. We therefore construct A5, which pursues the same substitution goal as A1 but is written against the deployed stack, and we decompose it into six independently switchable evasion techniques so that its success can be attributed:

E1 Query paraphrase. 

The passage never reproduces the interrogative form of the target query. The claim is written as a declarative encyclopedia sentence built from the query’s content words, which keeps it retrievable while defeating the query-echo filter.

E2 Assertive payload. 

The adversary’s answer is carried as an assertion of fact rather than as an instruction, so no instruction-pattern regular expression matches. Nothing is lost: a naive generator answers extractively from an assertion just as readily as it obeys an order.

E3 Natural padding. 

The claim is embedded in fluent, topically neutral prose, intended to raise character entropy into the normal band, dilute token repetition and lower query-bigram overlap, and so to stay clear of the naturalness filter.

E4 Similarity dilution. 

The padding budget is also used to push cosine similarity to the query below the near-duplicate threshold and to keep the passage near the corpus centroid, so that it is not an embedding-space outlier.

E5 Provenance spoofing. 

Documents are uploaded claiming the source identifier the deployment already trusts, with a correspondingly high trust score. This models a single compromised connector, a stolen ingestion credential, or an insider, and it satisfies both an allowlist and a trust threshold at zero cost, because both are assertions rather than proofs.

E6 Corroboration. 

Each of the adversary’s b documents is textually distinct and carries a distinct document identifier, so content-hash deduplication does not collapse them and the aggregator’s redundancy bonus (Equation (2)) counts them as independent corroboration of the false claim.

Algorithm 6 gives the construction. We stress that A5 uses no gradient information, no queries to the deployed system and no capability beyond those granted in Section 3: it is a rewriting adversary, cheap to implement and cheap to run.

Table 3 puts one crafted passage from each attack side by side. The comparison is the argument of this paper in miniature. Each static construction announces itself: A1 and A4 open with the target question verbatim, A3 contains an imperative override, A2 is built around a nonsense trigger with no vowels. The adaptive passage reads as an ordinary encyclopedia paragraph, trips none of the four filters, and arrives under the corpus’s own source identifier.

Algorithm 6 A5: adaptive poison construction

6. Experimental Design

6.1. Corpora

We evaluate on three corpora (Table 4). SQuAD [49] supplies encyclopedic paragraphs with one question each. HotpotQA [50] is used in its distractor configuration, contributing both supporting and distractor paragraphs; its high document-to-question ratio is deliberate, because a corpus in which every non-poisoned passage is topically unrelated would let a filter achieve a low false-positive rate for the wrong reason. A small bundled toy corpus is retained so that the entire study can be reproduced offline in seconds.

6.2. Models

Passages and queries are embedded with all-MiniLM-L6-v2 [21,51], a 384-dimensional sentence encoder, and indexed with exact inner-product search [22]. The generator is a deterministic mock model that answers extractively from the passage most lexically aligned with the question and, unless hardened, obeys instructions found in retrieved text. This is a deliberate instrument rather than a convenience: it fixes the generator’s behavior exactly, so that changes in measured attack success are attributable to the retrieval pipeline and the defense layers rather than to sampling noise or to an opaque commercial model’s undisclosed internal filtering. It models a worst-case susceptible generator; Section 7.10 reports the same experiments with instruction-following disabled, which bounds how much of the measured risk depends on that assumption. Section 9 discusses what this choice does not tell us.

6.3. Metrics

We report attack success rate (ASR), the fraction of attacked queries meeting the adversary’s goal from Section 3, with 95% Wilson score intervals [52] over attacked queries pooled across seeds. Wilson intervals are used rather than the normal approximation because the rates here sit routinely at or near 0 and 1, where the Wald interval degenerates to zero width and would misreport “no successes in n trials” as certainty [53].

For suppression attacks the unconditional ASR is inflated by the pipeline’s own errors: a query the system was going to get wrong anyway is counted as a successful suppression. We therefore also report attributable ASR, the success rate restricted to queries that the same configuration answered correctly in a clean run with no adversary present. Every configuration is run once without an attack precisely to establish this per-configuration baseline.

We further report clean accuracy and abstention rate (the utility cost on benign traffic); poison retrieval rate and poison use rate (whether poison reached the candidate set, and whether it supported the returned answer); the retrieval filter’s true- and false-positive rates at the deployed operating point, together with threshold-free AUROC and average precision. Because poisoned passages are a small minority of the candidate set, average precision is the more informative summary of detector quality [54,55]. Ground-truth poison labels are used exclusively for scoring; no defense component reads them. Latency is decomposed into retrieval, filtering and generation, and reported as mean and 95th percentile.

6.4. Configurations and Studies

Seven configurations are compared: the undefended pipeline; each layer alone (L1–L4); the full stack; and the full stack with signature-bound ingestion, in which the legitimate corpus is signed at the source and Layer 1 requires a valid signature. Unless stated otherwise $k=3$, $m=10$, ${\theta }_{\mathrm{trust}}=0.6$, and the adversary’s budget is one document per target for A1–A3, five for A4 and three for A5. Every cell is repeated over three seeds, which vary the decoy answers assigned to targets.

The studies reported in Section 7 are: the main grid (configurations × attacks × corpora); the evasion leave-one-out; the adversary budget sweep; the filter threshold sweep; the over-retrieval sweep; the ingestion-control ladder; and the generator-hardening comparison.

6.5. Reproducibility

The framework, the attacks, the defense layers, the experiment suite and the code that generates every table and figure in this paper are contained in a single repository. Results are written incrementally as keyed JSON lines, so an interrupted sweep resumes without recomputation, and every number reported here is regenerated from those files by a single command rather than transcribed.

7. Results

All rates are pooled over attacked queries from 3 seeds and reported with 95% Wilson intervals. Unless a subsection says otherwise, figures show SQuAD, and the corresponding HotpotQA and toy-corpus panels are included where they differ materially.

7.1. The Undefended Pipeline

Table 5 gives the full grid. Undefended, every attack succeeds. Pooled over the four static attacks, attack success is 0.95 on SQuAD and 0.95 on HotpotQA; the adaptive attack reaches 0.82 and 0.96 respectively.

Clean accuracy of the same pipeline is 0.62 on SQuAD but only 0.25 on HotpotQA. That gap is a property of the generator, not of the defense: the mock model answers extractively from the single most query-aligned passage, and HotpotQA questions require composing two. We report HotpotQA regardless, because its distractor paragraphs give the retrieval filter a realistic supply of topically adjacent non-answers, but its low benign ceiling is exactly why the attributable rate of Section 6.3 is needed: on that corpus, an unconditional suppression rate would be dominated by questions the pipeline never answered correctly in the first place.

7.2. Layer Ablation Against Static Attacks

Figure 2 and Figure 3 show what each layer contributes. Ingestion verification and retrieval filtering each independently reduce pooled static attack success on SQuAD to 0.00 and 0.01 in attributable terms, and the full stack to 0.00. They do so by different mechanisms: Layer 1 rejects the poison before it is indexed, so it is never retrieved, whereas Layer 2 admits it to the index and removes it from the candidate set.

Robust aggregation alone is markedly weaker, leaving 0.68 of static attacks successful. Its isolate-then-aggregate design defeats prompt injection completely, because an injected directive that reaches only one isolated generation cannot outvote the passages around it. It does not defeat targeted poisoning, where the poisoned passage is the most query-relevant evidence in the candidate set and therefore wins the weighted vote on its own merits. It also carries the largest utility cost of any single layer: clean accuracy falls from 0.62 to 0.52 on SQuAD, because on queries where the evidence is genuinely split the layer abstains rather than guessing.

By construction, monitoring alone does not change attack success at all (0.95); it raised alerts under every attack, which is what a detective control is for, but the ablation makes explicit that detection is not prevention.

7.3. Suppression Requires the Attributable Rate

Context flooding, whose interaction with over-retrieval is examined in Section 7.9, is the case where the unconditional rate misleads. Under the full defense its raw ASR on SQuAD is 0.49, which would suggest the availability attack half-succeeds. Restricted to queries the same configuration answered correctly with no adversary present, its attributable ASR is 0.00: the flooding attack does not, in fact, suppress a single answer that the defended pipeline would otherwise have given. The raw figure was measuring the pipeline’s own error rate, since a query the system was going to get wrong is scored as a successful suppression. Undefended, the same attack has an attributable ASR of 1.00. We therefore report both quantities throughout and use the attributable rate whenever a suppression adversary is involved.

7.4. The Adaptive Adversary

The adaptive adversary changes the picture completely. On SQuAD its attributable success rate is 0.83 against the undefended pipeline and 0.85 against the full four-layer stack. That is not a reduction but, within the interval, no change at all. Layer 2’s true-positive rate against it falls to zero (Table 6): the filter does not flag a single poisoned passage, while its false-positive rate on legitimate evidence is unchanged. The same filter attains high detection scores against every static attack (Section 7.7), so this is not a weak detector; it is a detector whose signals the adversary has simply declined to emit.

Layer 3 does not rescue the situation; it makes it worse. The adversary uploads several textually distinct documents making the same claim, so the aggregation layer sees the false answer corroborated by independent document identifiers and applies its redundancy bonus to the adversary’s group. On every corpus, enabling robust aggregation raises adaptive attack success relative to the undefended pipeline: from 0.83 to 0.88 on SQuAD, from 0.97 to 0.98 on HotpotQA, and from 0.41 to 1.00 on the toy corpus, where the effect is starkest because the true answer is supported by exactly one document and the adversary supplies three. The mechanism that makes isolate-then-aggregate robust to a corrupted minority is precisely the mechanism the adversary exploits: at a budget of three documents per target, the adversary simply buys the majority. This is not a tuning failure of our particular weighting; it is what any consistency- or corroboration-based aggregator must do when the adversary controls more of the evidence than the corpus does.

Only the ingestion signature holds. With Layer 1 requiring a valid HMAC over document content, the adaptive attack’s attributable success falls to 0.01 on SQuAD and 0.00 on HotpotQA, and clean accuracy is unchanged at 0.51. The poisoned documents are rejected at ingestion and never reach the index.

7.5. Which Evasion Technique Matters

Table 7 and Figure 4 disable one evasion technique at a time and leave the other five enabled. Three of the six are load-bearing and three are not.

Disabling provenance spoofing (E5) alone reduces attack success from 0.83 to 0.01: the poison is uploaded under an untrusted source name, the allowlist rejects it, and no content filter is needed. Disabling paraphrase (E1) reduces it to 0.00, because the passage then opens with the target question and the query-echo filter recovers. Disabling the assertive payload (E2) reduces it to 0.01, with the instruction-pattern filter’s true-positive rate returning to unity, since an imperative override is exactly what that filter was built to see.

The other three techniques are, against this stack, unnecessary. Removing natural padding, similarity dilution or cross-document corroboration leaves attack success at 0.83, 0.83 and 0.83 respectively, statistically indistinguishable from the full attack. The naturalness and similarity-outlier filters were running in all of these cells; they simply had nothing to fire on either way. A declarative sentence about the target topic, at ordinary length, is already fluent, already unrepetitive and already close to the corpus centroid, so padding it further changes nothing a detector can see. E3, E4 and E6 are therefore insurance on the adversary’s side, cheap protection against detectors this deployment happens not to have, rather than requirements against the one it does. Under signature-bound ingestion every variant fails regardless of which content-level technique is disabled (final column of Table 7), because the ingestion check does not depend on what the document says.

The middle columns of Table 7 separate the two mechanisms by which a technique can be load-bearing. With paraphrasing disabled the poison is still retrieved for every attacked query, and the poison-retrieval rate stays at unity, but the filter now flags enough of it to keep it out of the answer. With provenance spoofing disabled the poison-retrieval rate falls to zero: nothing is filtered because nothing was ever indexed. The first is detection after the fact, the second is prevention, and only the second leaves nothing for the adversary to iterate against.

The practical reading is uncomfortable for content-based defense. Two of the three necessary techniques are single rewriting decisions, namely writing a statement instead of a question and asserting instead of commanding, that cost the adversary nothing in effectiveness. The third is a credential problem, not a text problem.

7.6. The Ingestion-Control Ladder

Section 7.5 identifies provenance spoofing as necessary for the attack of Section 7.4; Table 8 and Figure 5 ask which ingestion controls actually stop it. Against a static adversary the ladder behaves as intuition suggests: adding a trust threshold takes success from 0.99 to 0.01, and an allowlist to 0.01. Against the adaptive adversary the same rungs do nothing at all (0.82, 0.82 and 0.82 respectively), because the source identifier and the trust score are attributes the uploader asserts, and the adversary asserts the ones the deployment already accepts.

The per-source quota is the one intermediate rung that does anything, and it repays a careful reading. We set it at $1.5$ times the size of the legitimate corpus, meaning that the contributing source may grow what it has already supplied by half. A quota expressed as a fixed document count does not measure a defense at all: set below the corpus size it silently truncates the knowledge base, and the resulting inability to answer is then reported as security.

Under that policy the quota reduces adaptive attack success to 0.15 on SQuAD, rejecting most of the adversary’s uploads, and to 0.96 on HotpotQA, where it rejects none of them. The difference is not a property of the defense but of the ratio between the campaign and the corpus. Attacking all 300 SQuAD questions with three documents each requires 900 uploads against a remaining budget of 150; the same campaign on HotpotQA is 600 uploads against a budget of nearly 1000, and fits. A quota is a volume control, not an authenticity control: it prices a mass campaign and is blind to a targeted one. Section 7.8 shows the adaptive attack is already effective at one document per target, so an adversary interested in a single high-value question never approaches any quota a growing corpus can tolerate.

Requiring a signature takes adaptive attack success to 0.01 while rejecting every poisoned document at ingestion. This is the single clearest result in the study, and it is not a machine-learning result: it is the observation that the distinction between a claimed and a proved attribute, long settled in software supply-chain security [41,42,43], has not yet been imported into retrieval pipelines.

7.7. Detection Quality and Operating Point

Table 9 reports threshold-free detector quality and Figure 6 the empirical ROC traced by sweeping the calibrated suspicion threshold $\tau$ of Equation (1). Both are measured in the filter-only configuration, because in the full stack Layer 1 rejects the static poison before it is ever a retrieval candidate and the filter’s score would otherwise be credited with Layer 1’s work.

Against the static attacks the filter is close to separable: AUROC is at least 0.99 on SQuAD, 0.97 on HotpotQA and 0.99 on the toy corpus, with average precision to match. Against the adaptive attack AUROC is 0.47, 0.65 and 0.32 respectively. These lie near chance, and on two of the three corpora below it, meaning the suspicion score ranks adaptive poison as slightly less suspicious than the legitimate passages it competes with. At the deployed operating point the corresponding true-positive rates are 0.00, 0.02 and 0.00.

This is worth stating plainly, because a detector reported only at its default threshold and only against published attack constructions would look excellent.

Figure 7 shows what moving $\tau$ costs. Detection of the adaptive attack can be bought, but not at a price any deployment would pay. At the default operating point $\tau =1.0$ the filter catches 0.00 of adaptive poison at a false-positive rate of 0.01, leaving attack success at 0.84 with clean accuracy 0.61. Tightening to $\tau =0.7$ raises adaptive detection to 0.21 and cuts attack success to 0.46, but clean accuracy has already collapsed to 0.25, because the filter is now discarding 0.09 of legitimate evidence. At $\tau =0.5$ attack success is 0.07, and clean accuracy is 0.06: the system has been secured by being switched off. Throughout the sweep the static attacks are caught at a true-positive rate of essentially one, so no single threshold serves both adversaries.

The reason is that the adaptive passages are not merely sitting below the threshold. They are indistinguishable on these features from genuine encyclopedic prose, which is what they were written to be, so the only threshold that captures them is one that captures the corpus too.

7.8. Adversary Budget

Figure 8 sweeps the number of documents b the adversary inserts per target. The undefended pipeline is saturated at a single document, consistent with the retrieval condition PoisonedRAG identifies: one passage constructed to embed next to the query is enough to enter a top-3 context. Under the full defense the static curve stays near zero across the whole range, so more poison does not help an adversary whose construction is detectable. Under signature-bound ingestion the adaptive curve is flat at zero: budget is irrelevant when every document is rejected before indexing.

The adaptive curve against the full defense has a shape worth reading closely, because it prices the attack. Against the undefended pipeline it is flat, at 0.83 for $b=1$ and 0.85 for $b=20$ on SQuAD, since one retrievable passage already carries the false claim. Against the defended pipeline it steps: 0.70 at $b=1$, rising to 0.85 at $b=2$ and flat thereafter. The step is starkest on the toy corpus, from 0.31 to 0.95.

That step is Layer 3’s redundancy bonus changing hands. With one poisoned document the adversary’s claim is a minority of one against the corpus’s genuine passage and the aggregator still resolves in favor of the truth a meaningful share of the time; with two, the false claim carries the corroboration bonus and the genuine passage does not. The adversary’s price for turning the aggregation layer from an obstacle into an instrument is therefore one additional document. Everything above $b=2$ is wasted effort, which is precisely the wrong shape for a defender: the attack that works leaves the smallest footprint, and volume-based monitoring has nothing to see.

7.9. Over-Retrieval, Utility and Latency

It is often argued, and we argued it ourselves when designing Algorithm 5, that a retrieval filter must be paired with over-retrieval, since dropping a candidate otherwise just shortens the context and hands a suppression adversary a win for free. Figure 9 tests that argument by sweeping the candidate multiplier m at two operating points, and the answer depends entirely on which one is deployed.

At the default threshold (left panel) the argument is simply void. Going from $m=1$ to $m=20$ leaves clean accuracy at 0.51, abstention at 0.31 and attributable static ASR at 0.00, identical to two decimal places, while p95 latency rises from 5 ms to 63 ms. Nothing is backfilled because nothing is dropped: with ingestion control in place the filter discards under one percent of candidates (0.00), so over-retrieval is pure cost. A deployment that sets a large multiplier for safety here is paying an order of magnitude in latency for no security and no utility.

Under strict filtering (right panel, $\tau =0.6$, false-positive rate 0.74 at $m=1$) the argument holds, but only for part of what it claims. Backfill repairs the availability damage convincingly: abstention falls from 0.67 to 0.04 and attributable static ASR from 0.25 to 0.03 as m grows. It does not repair utility: clean accuracy recovers only from 0.11 to 0.16, still far below the 0.51 obtained at the default threshold, because even at $m=20$ the filter is discarding 0.17 of legitimate evidence and no amount of over-retrieval manufactures evidence that was thrown away.

The practical conclusion is that backfill is a necessary companion to strict filtering and not a substitute for choosing the threshold correctly. The multiplier is the wrong lever: it can stop a badly tuned filter from denying service, but it cannot make a badly tuned filter useful.

Table 10 reports the utility cost of each configuration on unattacked traffic, Table 11 and Figure 10 the latency decomposition, and Figure 11 the resulting security–utility trade-off. The largest utility cost is Layer 3, which takes clean accuracy from 0.62 to 0.52 on SQuAD and raises the abstention rate from near zero to roughly a third of queries: on questions where the isolated answers disagree it declines to answer rather than picking one.

The measured time is dominated by filtering, at roughly 25 ms of a 26 ms defended query, because the layer scores $mk=30$ candidates with four detectors, one of which maintains corpus-level statistics. This ranking is an artifact of the harness and should not be carried into a deployment: the generator here is a local mock costing a fraction of a millisecond per call, so the one component that would dominate a real system is precisely the one this measurement makes invisible. The structural cost of Layer 3 is not a matter of measurement at all: by construction it issues one generator call per retained passage instead of one per query, so at $k=3$ it triples generator spend regardless of how fast the generator is. The honest reading of Table 11 is therefore that the defense’s own overhead is dominated by candidate filtering and scales with m, while its dominant deployment cost is the k-fold multiplication of generator calls.

7.10. How Much Depends on a Susceptible Generator

The generator used throughout obeys instructions found in retrieved text, which is a worst-case assumption. Table 12 and Figure 12 repeat the experiments with that behavior disabled. Prompt injection is, as expected, eliminated: with no instruction-following there is no instruction hierarchy to override. The other four attacks are essentially unaffected, including the adaptive attack, because none of them relies on the generator obeying an order. They rely on the generator believing a document, which any faithful, grounded generator will do by design. Indeed, a generator that ignored its retrieved context would defeat the purpose of retrieval. Instruction-hierarchy training and structured prompting [35,36,37] therefore address one of the five attacks in this study and leave the knowledge-integrity problem untouched.

8. Discussion

8.1. What the Layers Are Actually Doing

Read together, the ablation of Section 7 and the evasion study give each layer a more precise role than “defense in depth” usually implies, and one that the suppression correction of Section 7.3 is needed to see clearly.

Layer 1 is the only preventive control in the stack that an adaptive adversary cannot argue with, but only in its cryptographic form. Its allowlist and trust threshold check assertions, and an adversary who can write to the corpus can assert anything. This distinction is invisible against static attacks, which conveniently upload from an untrusted source name, and it is the whole result against an adaptive one.

Layer 2 is a detective control that has been widely deployed as though it were preventive. It is genuinely effective against the published attack constructions, and it generalizes to nothing beyond them: its four signals correspond to four things an adversary can simply not do. We do not conclude that retrieval filtering is useless, since it raises the cost of the naive attack and produces the evidence an incident responder needs, but it should be budgeted as detection, and a system whose security case rests on it does not have one.

Layer 3 (Section 4.3) is the layer whose behavior most needs restating. Against prompt injection it is the strongest single control in the stack, and its abstention is a genuinely useful failure mode: under attack it prefers to answer nothing over answering wrongly. But its robustness rests on the corrupted passages being a minority, and an adversary with a budget of three documents per target owns that majority. Our measurements show this is not merely a missed opportunity: on every corpus, enabling robust aggregation increased adaptive attack success, because the redundancy bonus that rewards corroboration rewards whoever supplies the most of it. It is also the most expensive layer in both utility (Figure 11) and compute (Figure 10). Certified variants make the underlying assumption explicit [13], which is the right response: the guarantee is conditional on a corruption bound, and a deployment has to enforce that bound somewhere else, namely at ingestion. A corroboration-weighted aggregator deployed without such a bound is not a neutral addition; it is an amplifier pointed in whichever direction the evidence majority lies.

Layer 4 does what detective controls do. It alerted under every attack, and it cannot stop any of them. Its value is operational, and our ablation is included precisely so that this is not confused with prevention.

8.2. Guidance for Deployments

Four recommendations follow from the measurements rather than from the architecture.

Sign at the source, verify at ingestion (Section 7.6). Bind each document to a key held by the ingestion pipeline, not to a source name in a metadata field. This is the only control in our study that survived, it costs one HMAC per document, and its clean accuracy cost was nil. Where content genuinely must be accepted from unauthenticated sources, treat that corpus as a separate, lower-trust index rather than mixing it with signed content.

Budget content filters as detection. Deploy them, alert on them, retain their scores for investigation, and do not let them appear in the security case as a preventive control. Report their performance against rewritten payloads, not only against published constructions.

Tune the filter’s threshold; do not compensate with over-retrieval. The two knobs are not interchangeable, and we found the intuitive advice to be wrong in the regime deployments actually run in. At a sensible threshold the filter discards so little that raising the candidate multiplier changes no outcome at all and multiplies latency roughly tenfold. At an strict threshold, over-retrieval does prevent the filter from denying service, since abstention and attack success both fall steeply with m, but it recovers only a fraction of the clean accuracy the threshold gave away. Set the operating point from a measured false-positive rate on legitimate traffic, use the smallest multiplier that keeps abstention flat, and treat a large multiplier as a symptom of a mistuned filter rather than a safety margin.

Do not conflate instruction-hierarchy defenses with knowledge integrity. Hardening the generator against injected instructions eliminated exactly one of our five attacks. The rest attack what the model is given to believe, and a well-behaved grounded generator will believe it.

8.3. Why Content-Based Detection Fails Here

The failure is structural rather than a matter of tuning. Each of our detectors keys on an artifact of a convenient attack construction: the query is echoed because that is the easiest way to guarantee retrieval; the payload is imperative because that is the easiest way to steer a generator; the text is repetitive because it was templated. None of these is required. A passage that asserts a false fact in fluent prose, on topic, at ordinary length, is by construction indistinguishable from a passage that asserts a true fact in fluent prose. The difference is the truth of the claim, which is not a property of the text’s surface form. This is why the adaptive ROC in Figure 6 sits on the diagonal, and why we expect detectors built on richer surface features, including perplexity-based ones [39,40], to face the same ceiling. Distinguishing true from false claims is a knowledge problem, and the pipeline’s answer to knowledge problems is to consult the corpus, which the adversary controls.

8.4. Relation to Prior Results

Our static-attack results reproduce the qualitative findings of the poisoning literature: a single crafted passage is enough to hijack a targeted query in an undefended pipeline [5,6], and availability-style poisoning suppresses answers without substituting one [12]. Our defended results are consistent with reported gains for retrieval-time detection and isolate-then-aggregate generation [13,14] when evaluated the same way those works evaluate. The divergence appears only under the adaptive adversary, which is the predicted consequence of the methodological gap that Carlini and Wagner [16], Athalye et al. [17], Tramèr et al. [18] documented in a different subfield. We read our results not as a refutation of those defenses but as evidence that the RAG security literature is at the point in its development where adaptive evaluation must become standard.

9. Limitations

The generator is a deterministic mock. This is deliberate, since it isolates the retrieval pipeline and makes attack success measurable without an opaque model in the loop, but it is not a production language model. A stronger model might resist a single false assertion that contradicts its parametric knowledge, and the generator study bounds only the instruction-following component of that question. Replicating this study across current hosted and open-weight models is the most important next step, and we expect the adaptive attack’s advantage to shrink but not vanish, since a model asked to ground its answer in a corroborated retrieved claim has little basis for refusing.

Our corpora are hundreds of documents, not the millions of an enterprise index. Scale affects the similarity-outlier filter, which estimates corpus statistics, and it affects how easily a poisoned passage reaches the top-k for a query. It does not obviously affect the central result, since provenance spoofing is scale-independent, but this is an assumption rather than a measurement.

The adaptive adversary is one adversary, constructed by us against our own defense. It gives a lower bound on what an adaptive attacker achieves, not an upper bound, and a stronger one almost certainly exists, in particular one that optimizes passage embeddings against the retriever directly rather than relying on lexical overlap. Symmetrically, our defense is one implementation; a deployment with different filters would need its own adaptive evaluation, which is the point.

Attacks and defenses were implemented in the same codebase, which risks benchmark coupling. We mitigated this by exposing evasion techniques as independent switches and reporting per-layer attribution rather than aggregate scores, so a reader can see which defensive assumption each result depends on, but independent reimplementation would be stronger evidence.

Finally, reported latencies exclude external model and network cost, and the abstention behavior we report is a property of our aggregation thresholds rather than a calibrated operating point that a deployment should adopt unexamined.

10. Conclusions

We built a four-layer defense against knowledge-base poisoning in retrieval-augmented generation (ingestion verification, retrieval filtering, robust aggregation and anomaly monitoring) and evaluated it on three corpora against five attacks, including a defense-aware adaptive adversary of our own construction.

Against the four static attacks from the literature the architecture performs as expected: attack success falls from near unity to approximately zero in attributable terms, at a modest utility cost. Against an adversary who merely rewrites the payload, making it declarative instead of interrogative and assertive instead of imperative, and uploading it under a source name the deployment already trusts, the same architecture fails. The retrieval filter’s true-positive rate falls to near zero and its ROC to the chance diagonal; robust aggregation is turned against itself by manufactured corroboration; and attack success remains close to that of the undefended pipeline, and on one corpus far above it. A leave-one-out ablation shows the adversary needs exactly three of its six techniques, two of them free rewriting choices.

Requiring an unforgeable signature at ingestion restored the defense completely, at no cost to clean accuracy, and was the only control in the study that did. We take this as the practical conclusion: content-based heuristics belong in a RAG security architecture as detection and as incident evidence, but the preventive control has to be verifiable provenance. The security question for a retrieval pipeline is not primarily what a document says, since an adversary chooses that, but whether the pipeline can prove who put it there.