Preprint
Article

This version is not peer-reviewed.

Faithful but Wrong: Validating LLM Faithfulness Judges for Retrieval-Augmented Generation Under Corrupted Context

Submitted:

15 November 2026

Posted:

17 November 2026

You are already at the latest version

Abstract
LLM-judged faithfulness is widely used to certify RAG answers, yet it is rarely checked against human labels, and it is unclear what it rewards when the retrieved context is wrong. We measure faithfulness, correctness, refusal, and adoption of injected falsehoods on the same answers under controlled context perturbations, and audit the faithfulness judge against two human annotators. From SQuAD v2.0 we build 75 answerable and 20 unanswerable questions, with six context conditions for answerable questions (clean, random distractors, related distractors, heavy noise, contradictory evidence, answer-absent) and three prompt mitigations. Three open-weight generators (Qwen3-8B, DeepSeek-R1-Distill-Qwen-32B, Llama-3.1-8B) produce 4,230 context-conditioned answers, scored with Ragas by a local 27B judge and rule-based checks, plus two cross-family API judges (Gemini, Claude). For Qwen3-8B, distractors had no large effect (equivalent to zero within +/-10 points under lenient matching). In all three models, a contradictory passage was the damaging perturbation: under baseline and refusal prompts, correctness fell by 20-35 points and models repeated the injected falsehood in 23-48% of answers, almost never flagging the conflict. The local judge rated 86-94% of these answers faithful. Human annotators agreed closely with each other (kappa = 0.96 on supported versus not supported). Re-weighted within each condition, the local judge agreed with humans on 91-92% of Qwen3-8B answers (kappa = 0.62-0.69), but its errors concentrated in its unfaithful verdicts. Gemini and Haiku had numerically higher agreement on the same answers, though not significantly so. A judge that mostly agrees with humans about grounding can still certify a wrong answer, so faithfulness judgments cannot stand in for correctness. The effect of prompt mitigations depends on the model.
Keywords: 
;  ;  ;  ;  ;  

1. Introduction

Retrieval-augmented generation (RAG) conditions a language model’s answer on passages retrieved from an external corpus [1,2], and it is now a default architecture for question answering over private or changing documents [3,4,5]. Its promise is that answers will be grounded in the retrieved text. In practice, retrieval and grounding fail independently: the retriever may return irrelevant or incomplete passages, the generator may ignore relevant passages, and passages may conflict with each other or with the model’s parametric knowledge [6,7,8]. Commercial retrieval-based legal tools hallucinated on 17–33% of queries despite vendor claims [9].
Prior work has approached these failures from two directions. Robustness benchmarks such as RGB [6] and ClashEval [10] perturb the context with noise, missing answers, or counterfactual values and report answer accuracy [7,8,11,12,13], while evaluation frameworks such as Ragas [14], TruLens [15], and RAGChecker [16] score faithfulness and relevance, usually on natural retrieval and with the judge taken on trust. Our contribution is to combine the two: we measure faithfulness, correctness, and refusal together on the same answers under graded, controlled perturbations, and we audit the faithfulness judge itself against human annotators and against judges from other model families, in a local setup with three open-weight generators, including a reasoning model.
Existing work rarely evaluates robustness and faithfulness on the same answers, and rarely audits the faithfulness judge itself. We address this gap with four research questions. RQ1: How do faithfulness, context quality, and answer relevance change as context degrades from clean to noisy, contradictory, and answer-absent? RQ2: Do perturbation types differ in effect, and does faithfulness agree with correctness under each? RQ3: Which prompt-level mitigations help, and at what cost in unnecessary refusals? RQ4: Do these findings hold across generators and across judges?
Our contributions are: (i) a joint evaluation of faithfulness, correctness, refusal, and falsehood adoption on the same answers, which separates faithful-but-wrong from correct-but-ungrounded answers; (ii) a two-annotator audit of the LLM faithfulness judge, including a prevalence-corrected agreement estimate, an analysis of where its verdicts fail, and a comparison with cross-family judges; (iii) a paired comparison of three prompt-level mitigations with their refusal trade-offs, showing that their effect depends on the generator; (iv) a replication across three open-weight generators, including a reasoning model that adopts falsehoods less often but sometimes never terminates its reasoning under contradiction; and (v) a reproducible perturbation protocol over SQuAD v2.0 with leakage-free distractors, hand-reviewed contradictions, and position randomization.

3. Method

3.1. Data, Corpus, and Retrieval

We use the SQuAD v2.0 validation split [33]. All paragraphs of 25 randomly sampled articles form an 861-passage corpus. We select 75 answerable questions (gold answer of at most five words, not contained in the question, at most one question per paragraph), keeping all annotator answers as gold, and 20 unanswerable questions from the same articles. Passages are embedded with nomic-embed-text [34] and indexed in Chroma with cosine similarity; the gold passage ranks in the top four for 95% of answerable questions. Retrieval supplies related distractors, while gold passages are placed explicitly so that every context’s composition is known.

3.2. Context Conditions

Each answerable question receives six contexts (Table 1); unanswerable questions receive only C5 (their source paragraph plus three related passages), giving 470 contexts. Distractors exclude any passage that contains a gold answer under exact or lenient matching, so C5 contains no answer and distractors are uninformative. For C4, every occurrence of the gold answer in a copy of the gold passage is replaced with a same-type substitute: numbers are shifted with format preserved, and other answers are drawn from SQuAD answers to questions with the same question word. All 75 substitutes were reviewed by hand and 55 replaced with more plausible alternatives; each is automatically checked not to appear in the gold passage or to match a gold answer. Passage order is shuffled with a fixed seed, and gold and conflict positions are recorded.

3.3. Mitigations and Generation

Each context is presented under three prompts. M0 (baseline): “Use the following context to answer the question.” M1 (strict): answer using only the context and reply exactly “I don’t know” if it lacks the answer. M2 (context-faithful) [29]: the passages are quoted as “Bob said: …”, the question asks for Bob’s opinion, two counterfactual demonstrations follow Bob’s false statements, and the M1 refusal rule is included so that M1 and M2 differ only in framing. All prompts request a one-sentence answer. A closed-book prompt without context is run once per answerable question as a rough probe of parametric knowledge; because many SQuAD questions presuppose their paragraph, it gives a lower bound. We audit three open-weight generators served by Ollama [35] (all Q4_K_M), decoding at temperature 0 with seed 42: Qwen3-8B [36] with thinking disabled; DeepSeek-R1-Distill-Qwen-32B [37] (abbreviated R1-32B in tables), a reasoning model distilled into a Qwen2.5 base, with reasoning enabled (it cannot be disabled without the reasoning leaking into the answer), a budget of 4,096 output tokens including reasoning, and the reasoning trace stored separately from the answer; and Llama-3.1-8B-Instruct [38], a different model line. Qwen3-8B is the primary model and receives the full analysis; the other two test whether the findings generalize. Each generator yields 470 × 3 = 1 , 410 context-conditioned and 75 closed-book answers; across the three generators, this gives 4,230 context-conditioned and 225 closed-book answers. An answer left empty because the reasoning exhausted its budget is counted as incorrect and not as a refusal.

3.4. Measures

LLM-based scoring uses a 27.3B Qwen-family judge run locally in Ollama (local tag qwen3.8:27b; architecture qwen35, Q4_K_M, digest 22130167c4c2) at temperature 0 with reasoning disabled, keeping the primary scoring offline; it scores all three generators, so their scores are comparable. The judge shares a model family with Qwen3-8B and with the Qwen2.5 base of the distilled reasoning model, but not with Llama. To test whether judge findings are specific to this family, two API judges from other families score a subset of Qwen3-8B answers (the human-labeled sample and all C4 answers) and the C4 answers of the other two generators, using identical Ragas prompts: Gemini 3.8 Flash [39] at temperature 0 with reasoning disabled, and Claude Haiku 4.5 [40], run without a temperature setting, so it used its default sampling. Answer relevance uses the same local embedding model for every judge. Faithfulness (Ragas) is the share of an answer’s atomic statements supported by the passages; it is not computed for refusals. Answer relevance (Ragas) is reported for non-refused answers only, since it penalizes correct refusals. Context precision is the share of a context’s passages that the judge rates as helpful for the question; it replaced Ragas context relevance, which scored every C0–C4 context 1.0 in a pilot. Correctness is rule-based: after SQuAD normalization, an answer is correct if, for some gold answer, it contains every number and at least two-thirds of the content words (a strict exact-containment variant is also computed); in C5, refusal is the correct behavior. Refusal is detected with a fixed phrase list, and falsehood adoption (C4) applies the correctness rule to the substituted answer. M2 answers begin “In Bob’s opinion”; because the judge rated such answers unfaithful, this framing is removed before judging.

3.5. Human Validation and Statistics

We sample 80 Qwen3-8B answers: 12 non-refused answers per condition, balanced across mitigations and between judge-faithful ( ≥ 0.5 ) and judge-unfaithful answers, plus 8 refusals. Two annotators, one of them an author, independently label each answer for support (supported, partial, unsupported), correctness, and refusal without seeing judge scores or each other’s labels. One sampled answer that the refusal rule missed was labeled a refusal by both annotators, leaving 71 answers for support comparisons. We report Cohen’s κ  [41] and Spearman’s ρ . Because κ depends on the sampling design and the sample over-represents the judge’s rare “unfaithful” verdicts, we also re-weight the per-verdict human rates within each condition, using the run’s judge-faithful share for that condition, which yields prevalence-corrected estimates of agreement and of the share of human-supported answers. The sample is stratified by condition, so the per-condition estimate is the headline; a pooled version, which re-weights across all conditions, is reported as a sensitivity check (Section 4).
Means are reported with 95% bootstrap confidence intervals over questions. Because every question appears in every cell, comparisons are paired: exact McNemar tests [42] for binary outcomes and Wilcoxon signed-rank tests [43] for faithfulness, with Holm adjustment within each family of comparisons. For the noise conditions we additionally report bootstrap confidence intervals on paired differences and test equivalence within ± 10 points (two one-sided tests, i.e., the 90% interval inside the margin). Position effects use χ 2 and Kruskal–Wallis tests, and the link between prior knowledge and falsehood adoption uses Fisher’s exact test (exploratory, Holm-adjusted across mitigations). A 10-question pilot preceded the study and led to the context-precision measure, the lenient correctness rule, stricter distractor filtering, hand-reviewed substitutes, and M2 framing removal; all were fixed before the full run. To check that these choices do not drive the conclusions, we repeat the correctness analyses with the strict rule.

4. Results

Sections on RQ1–RQ3 analyze Qwen3-8B in full; Table 2 and Figure 1 summarize its results, and p-values are Holm-adjusted unless noted. RQ4 compares all three generators.

4.1. RQ1: Context Degradation

Context precision fell from 0.99 (C0) to 0.25 (C1), 0.33 (C2), 0.19 (C3), and 0.12 (C5), confirming that the passage-level judgment registers the injected noise; C4 scored 0.56 because the contradictory copy is topically relevant. The answers, however, showed no large effect of noise: correctness in C1–C3 (0.91–0.95) stayed within a few points of C0 (0.93–0.95), with no significant difference (all unadjusted p ≥ 0.25 ). Because correctness is near ceiling, a null test alone is weak evidence, so we tested equivalence: all nine C0-versus-noise and three C1-versus-C2 differences were equivalent to zero within ± 10 points (largest 95% interval bound − 9.3 points). Under the strict rule, nine of twelve were equivalent; the exceptions (C0 vs. C3 under M0, C0 vs. C2 under M2, C1 vs. C2 under M1) had point estimates of − 5 to − 7 points and lower 95% bounds near − 13 points, so moderate drops in exact-match accuracy cannot be excluded. Faithfulness differences were also equivalent within ± 0.10 , although under M0 the decline from C0 to C3 ( − 0.044 , 95% CI [ − 0.087 , − 0.011 ] ) excluded zero. Answer relevance was flat (0.90–0.92) across C0–C4. C4 was the only damaging perturbation for answerable questions: correctness fell from 0.947 to 0.707 under M0 (19 vs. 1 discordant questions, p = 0.001 ), and to 0.667 and 0.587 under M1 and M2 (both p < 0.001 ), while faithfulness stayed at 0.88–0.93 (all p = 1.0 ). With strict correctness the drop persisted (M0: 0.920 to 0.640; all three mitigations p ≤ 0.002 ). In C5, faithfulness of non-refused answers dropped to 0.29–0.42 while answer relevance stayed at 0.86–0.89: the answers addressed the question but were not grounded.

4.2. RQ2: Perturbation Type, Faithfulness, and Correctness

Random and related distractors (C1 vs. C2, equal length) did not differ in correctness or faithfulness under any mitigation (all unadjusted p ≥ 0.50 ). Table 3 cross-tabulates faithfulness against correctness. In C0–C3, 86–90% of answers were faithful and correct and only 4–6% faithful but wrong; in C4, 29% of judged answers were faithful but wrong (65 of 222; 34%, or 76 of 222, under the strict rule), because the model repeated the contradictory passage. This is expected by construction: the substituted copy is near-identical to the gold passage, so an answer drawn from it is grounded. The arguably correct behavior is to flag the conflict, and only one of 225 C4 answers did. Of 75 C4 answers under M0, 48 gave only the gold answer, 20 only the falsehood, 5 both, and 2 neither (Figure 2); under M2, falsehood-only answers rose to 28. The model answered 32% of questions correctly without context, a lower bound on its knowledge because many SQuAD questions presuppose their paragraph. In an exploratory analysis, M0 adopted the falsehood for 17% of questions known closed-book versus 41% of unknown ones (Fisher p = 0.040 , Holm-adjusted p = 0.12 ), with the same but weaker direction under M1 and M2; we treat this as suggestive only.

4.3. RQ3: Mitigations and Refusal

With the gold passage present, mitigations made little difference: no paired comparison in C0–C3 was significant, and unnecessary refusals across C0–C4 were 0% (M0), 2.1% (M1), and 0.5% (M2). With the answer absent, the refusal rule mattered (Figure 3). M0 refused only 7.4% of C5 questions; M1 raised this to 54.7% (45 vs. 0 discordant, p < 0.001 ) and M2 to 41.1% ( p < 0.001 ); M1 exceeded M2 (16 vs. 3, unadjusted p = 0.004 , adjusted p = 0.16 ). Refusal rates were 0/35/35% for truly unanswerable questions and 9/60/43% for gold-removed questions (M0/M1/M2). In C4, M2 had the lowest correctness (0.587 vs. 0.707, unadjusted p = 0.064 ) and the highest falsehood adoption (0.413 vs. 0.333), directionally consistent with its instruction to trust the passage but not significant after adjustment. Correctness by gold position (first 0.90, middle 0.85, last 0.86) did not differ ( χ 2 = 2.75 , p = 0.25 ), so we find no “lost in the middle” effect [13] at these context lengths.

4.4. RQ4: Generalization Across Generators

Table 4 compares the three generators on the rule-based measures, which need no judge. The central result replicates. Distractors left correctness high for all three (C1–C3 within 8 points of C0 under M0 and M1; Table 5), although Llama under M1 lost 7–8 points in C2 and C3 (88% and 87% against 95% in C0; not tested for equivalence), and the contradictory passage was the damaging perturbation in every model: under M0, correctness fell from 0.95–0.97 in C0 to 0.71–0.75 in C4 (exact McNemar, Holm-adjusted p < 0.001 for each model, and for every mitigation except Llama under M2, where refusals floor both conditions). Falsehood adoption under M0 and M1 was 33% for Qwen3-8B, 23–28% for the reasoning model, and 45–48% for Llama. The faithful-but-wrong pattern also replicates. The local judge rated 94% of the reasoning model’s falsehood-adopting C4 answers faithful (mean 0.94, n = 62 ), against a mean of 0.91 for its C4 answers that gave only the gold answer; for Qwen3-8B the figures were 86% (mean 0.83, n = 80 judged, non-refused answers; 81 adopted the falsehood in total) and 0.93. For Llama, 90% were rated faithful (mean 0.82, n = 78 ), against a mean of 0.90 for its gold-only answers. Faithful-but-wrong answers made up 29%, 19%, and 27% of judged C4 answers (empty answers excluded) for Qwen3-8B, the reasoning model, and Llama. Without the answer in context, the judge rated 41%, 56%, and 61% of non-refused C5 answers faithful (means 0.34, 0.49, and 0.57); given the judge’s tendency to overrate C5 answers, the higher values for the other two generators likely overstate their grounding. Llama’s answers also received lower answer relevance (0.72–0.86 across C0–C4, against 0.90–0.92 for Qwen3-8B and 0.90–0.94 for the reasoning model).
The reasoning model failed differently. On 4%, 11%, and 16% of C4 contexts (M0, M1, M2) and 8% of C5 contexts under M0, it spent its entire 4,096-token budget weighing the conflicting passages (“Wait, but the context also says…”) and produced no answer; its median output, reasoning included, was 254 tokens, so this non-termination is specific to conflicting or missing evidence. It adopted falsehoods less often than Qwen3-8B but did not flag conflicts more often. Its reasoning traces contained “conflict” or “contradict” in 96 of 225 C4 contexts (43%, against 13 of 225 C0 contexts, 6%), and 82 of those 96 went on to a non-empty answer. None of its 225 C4 answers flagged the conflict (one used the word “conflict” for a battle), against one flagged answer for Qwen3-8B and none for Llama. The model registered the contradiction in its reasoning but did not surface it in the answer.
Mitigation effects depended on the generator. Without a refusal rule, Qwen3-8B answered 93% of answer-absent questions, Llama 73%, and the reasoning model 65% (62 of 95; 25 refused and 8 empty). The refusal rule (M1) raised appropriate refusals to 55%, 61%, and 79% (Qwen3-8B, R1-32B, Llama), at a cost of 2.1%, 0%, and 8.5% unnecessary refusals on answerable questions. Context-faithful prompting (M2) behaved very differently across models: for Qwen3-8B it was comparable to M1, for the reasoning model it gave the most appropriate refusals of its three prompts (67%) at almost no cost (0.3%), and for Llama it was harmful, causing “I don’t know” on 79% of answerable questions, including 76% of clean C0 contexts where the gold passage was the only passage. Llama’s low falsehood adoption under M2 (11%) is therefore a by-product of refusing, not of resisting the falsehood. Closed-book accuracy was similar across models (32–40%).

4.5. Validity of the LLM Judge

The two annotators agreed closely: κ = 0.83 on the three-level support label (66 of 71), 0.96 on supported versus not (70 of 71), 0.95 on correctness, and 1.00 on refusal. Against these labels, the rule-based checks were reliable (correctness κ = 0.92 – 0.97 ; refusal κ = 0.93 ), but on the verdict-balanced sample the judge agreed only fairly on faithfulness ( κ = 0.36 and 0.33 ; Spearman ρ = 0.49 and 0.47 ). This sample over-represents the judge’s “unfaithful” verdicts, which make up only 15% of non-refused answers in the run (6–9% in C0–C4). Re-weighted within each condition to the run’s verdict mix, judge–human agreement was 0.92 and 0.91 for the first and second annotator ( κ = 0.69 and 0.62 ), i.e., substantial overall. Pooling the sample across conditions instead gives 0.84 for both annotators ( κ = 0.37 and 0.35 ). The difference arises because 111 of the run’s 201 unfaithful verdicts (55%) are in C5, where the annotators mostly confirmed the judge, but only 6 of the sample’s 36 unfaithful verdicts come from C5. Disagreement was asymmetric (Table 6): when the judge rated an answer faithful, the first annotator agreed in 32 of 35 cases, but 20 of 36 answers rated unfaithful were fully supported. Two patterns explain most errors. First, the judge penalized premises copied from the question: “The attacks resonated most with conservative Muslims” scored 0.0 in all 14 contexts where it appeared, because the passage says only “resonated with,” and “the first Huguenot church was L’Église française à la Nouvelle-Amsterdam” scored 0.0 although the name appears verbatim. Second, the judge used world knowledge: it scored 16% (11 of 68) of falsehood-only C4 answers as unfaithful versus 6% (8 of 134) of gold-only answers, among non-refused judged C4 answers pooled over mitigations (a 10-point difference, Newcombe 95% CI 1.5 to 21; two-sided Fisher’s exact p = 0.02 , descriptive only because answers to the same question are not independent), although both annotators labeled such answers supported.
Re-weighting the balanced sample to the whole run within each condition, the estimated share of answers a human would rate supported is approximately 1.00 (C0), 0.95–0.96 (C1–C3), 0.99 (C4), and 0.16–0.26 (C5, depending on the annotator), where the human rate is the share of answers rated fully supported. The like-for-like judge quantity is the share rated faithful (score ≥ 0.5 ) in the run: 94% (C0), 91–92% (C1–C3), 91% (C4), and 41% (C5). In C0–C4 the judge’s faithful share is 4–8 points below the human share; in C5 it is 15–25 points above it. The direction holds, but the gap in C0–C4 is smaller than a comparison of human shares with the judge’s mean score would suggest. For the first annotator, bootstrap 95% intervals are 0.92–0.99 for C1–C3, 0.96–1.00 for C4, and 0.00–0.33 for C5; the C0 interval is degenerate because all twelve sampled answers were supported. Each condition has only six judge-unfaithful sampled answers, so these estimates are approximate. The judge thus underestimates faithfulness when the gold passage is present and overestimates it when the answer is absent: it separates C5 from the other conditions in aggregate, but rates 41% of non-refused C5 answers as faithful, against a human-supported rate of 16–26%. No main conclusion changes; the faithful-but-wrong result in C4 is strengthened, and the small C0–C3 faithfulness decline (0.94 to 0.90) is within the judge’s error.
Cross-family judges. Table 7 compares the local judge with Gemini and Claude on the same human-labeled answers, and reports how each judge scores C4 answers that adopted the falsehood. Against the two annotators, both cross-family judges had numerically higher agreement with humans than the local judge did (no significance test was run): Gemini reached accuracy 0.73–0.75 and κ 0.37–0.40, Claude Haiku 0.76–0.78 and κ 0.47–0.50, versus 0.66–0.68 and 0.33–0.36 for the local judge. These comparisons use the 71 Qwen3-8B answers, because the human labels cover only that model. On the same answers, Gemini and Haiku agree with the local judge ( κ 0.61 and 0.62) and with each other less well ( κ 0.42). Both cross-family judges still share the central failure: Gemini rated 89%, 89%, and 90% of falsehood-adopting C4 answers as faithful for Qwen3-8B, R1, and Llama, and Haiku rated 69%, 86%, and 77%. Haiku is therefore less lenient on Qwen3-8B answers, but a majority of falsehood-adopting answers pass for every generator. The API judges scored only the C4 subset for R1 and Llama, and we did not test premise sensitivity or knowledge leakage separately for them, so we cannot say whether they share those mechanisms.

5. Discussion

Contradictions, not noise, are the risk. Unlike earlier findings that related distractors are especially harmful [11,12], for Qwen3-8B we detected no large effect of either distractor type when the gold passage was present (equivalent within ± 10 points under lenient matching). Llama under M1 lost 7–8 points in C2 and C3, which we did not test for equivalence. We hypothesize that this is because our noisy contexts are short (4–7 passages) and the distractors offer no competing answer, but we did not vary context length, so this remains untested. A passage that does offer a competing answer (C4) was far more damaging, in all three generators. Explicit reasoning did not resolve the conflict: the distilled reasoning model adopted the falsehood less often, but rather than flagging the conflict it sometimes deliberated until its budget ran out, a failure that a deployment with a token limit would surface as a silent non-answer.
Faithfulness is necessary but not sufficient. Answers that repeated injected misinformation were as faithful as correct answers, by both the judge and the annotators. This follows from the definition of faithfulness [14,17] and from our construction, in which the corrupted passage is near-identical to the gold one; the result is therefore a demonstration rather than an estimate of how often corrupted sources occur in practice. What it shows is that a groundedness metric cannot penalize such answers, and that Qwen3-8B almost never surfaced the conflict, which would have been the appropriate behavior. In C5 the judge’s mean faithfulness was low, so faithfulness does separate answer-absent hallucinations from grounded answers in aggregate, although the judge still rated many individual C5 answers as faithful (Section 4). Faithfulness metrics thus detect many unsupported answers but not corrupted sources; the latter require cross-source or closed-book consistency checks.
Refusal needs an explicit rule, and mitigations must be validated per model. Consistent with RGB [6], all three models usually answered without the answer in context. An explicit refusal rule helped every model, yet 21–45% of answer-absent questions were still answered, and its cost on answerable questions ranged from 0% to 8.5%. The same context-faithful prompt was neutral for one model, the best option for another, and made the third refuse most answerable questions, so a mitigation tuned on one generator should not be transferred to another without re-testing. For Qwen3-8B, context-faithful prompting [29], designed for cases where the context is right and memory is stale, refused less than the strict rule and adopted more injected falsehoods, echoing the receptiveness to coherent counter-evidence reported by Xie et al. [8]; it should be reserved for trusted collections.
LLM judges need calibration against humans. Our judge findings apply to one configuration: a same-family 27B judge with reasoning disabled, scoring through Ragas statement decomposition. For that configuration, overall agreement with humans was substantial once re-weighted within condition to the run (accuracy 0.91–0.92, κ = 0.62 – 0.69 ; 0.84 pooled), but the errors were concentrated in its rare “unfaithful” verdicts. With inter-annotator agreement of κ = 0.96 on supported versus not supported, these errors reflect the judge rather than task difficulty, and they add premise sensitivity and knowledge leakage to known judge biases [24]. Both cross-family judges had numerically higher agreement with the annotators on the same 71 answers (Table 7), but we did not break their errors down by verdict type, so whether they share the unfaithful-verdict errors remains unresolved. Whether trained grounding checkers [25,26] or reasoning-enabled judges share them remains open. In practice, low judge scores should trigger review, not be counted as hallucinations.
Limitations. We audit three open-weight generators at temperature 0 on 95 SQuAD questions whose Wikipedia source the models have likely seen. Two of the three share a Qwen lineage with the primary judge, and the generators differ in size and reasoning as well as family, so differences between them cannot be attributed to any one factor. The full statistical analysis (equivalence tests, position effects, prior knowledge) is reported for Qwen3-8B only. The reasoning model’s non-answers depend on its 4,096-token budget, which we did not vary. Claude ran at its default sampling because no temperature was set for it, and the API judges score only subsets of answers. The M2 prompt was refined while inspecting Qwen3-8B outputs in the pilot. C0 is shorter than the other conditions, so C0 comparisons mix noise with context length (C1 and C2 are length-matched). Contradictions are constructed rather than natural. Closed-book accuracy underestimates parametric knowledge, so the prior-knowledge analysis is exploratory. The rule-based checks can miss paraphrases. Human labels cover 80 Qwen3-8B answers from two annotators, one an author; κ on the balanced sample depends on its design, so the re-weighted estimates are approximate. Several analysis choices (context precision, the lenient rule, M2 framing removal) were made after the pilot; they were fixed before the full run, and the strict-rule analysis leaves the C4 result and the absence of large noise effects intact. Small differences between mitigations are not resolvable at this sample size.

6. Conclusions

A controlled audit of local RAG pipelines found no large effect of distractors on Qwen3-8B, whereas a near-identical contradictory passage cut correctness by 24–27 points for Qwen3-8B under M0 and M1 (36 under M2), and by 20–35 points across all three generators under M0 and M1 without reducing faithfulness and was almost never flagged, and the baseline prompt answered 93% of answer-absent questions (Qwen3-8B). Across three open-weight generators, the contradiction was consistently the damaging perturbation, the local judge rated 86–94% of falsehood-adopting answers faithful, a reasoning model traded some falsehood adoption for non-termination, and the effect of prompt mitigations depended on the model. Human validation confirmed the rule-based checks; the local faithfulness judge agreed with humans on most answers, but its “unfaithful” verdicts were unreliable in this configuration. Both cross-family judges had numerically higher agreement with the annotators than it did, but both still rated most falsehood-adopting answers as faithful. Practitioners should pair faithfulness metrics with refusal instructions validated for their model, source-consistency checks, and human review of flagged answers. Future work will cover larger and more diverse model families, sampling variability, and verification-based mitigations [22,30]. Code, data, prompts, and all model outputs will be released publicly.

GenAI Usage Disclosure

Large language models are the object of study: Qwen3-8B, DeepSeek-R1-Distill-Qwen-32B, and Llama-3.1-8B generated the evaluated answers, and a local Qwen-family model, Gemini 3.8 Flash, and Claude Haiku 4.5 served as automatic judges, as described in the Method section.

References

  1. Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Küttler, H.; Lewis, M.; tau Yih, W.; Rocktäschel, T.; et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2020. [Google Scholar]
  2. Gao, Y.; Xiong, Y.; Gao, X.; Jia, K.; Pan, J.; Bi, Y.; Dai, Y.; Sun, J.; Wang, M.; Wang, H. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv 2024, arXiv:cs.CL/2312.10997. [Google Scholar]
  3. LangChain. LangChain Documentation. Available online: https://python.langchain.com/.
  4. LlamaIndex. LlamaIndex Documentation. Available online: https://docs.llamaindex.ai/.
  5. Chroma. Chroma Documentation. Available online: https://docs.trychroma.com/.
  6. Chen, J.; Lin, H.; Han, X.; Sun, L. Benchmarking Large Language Models in Retrieval-Augmented Generation. In Proceedings of the Proc. AAAI Conf. Artificial Intelligence, 2024. [Google Scholar]
  7. Longpre, S.; Perisetla, K.; Chen, A.; Ramesh, N.; DuBois, C.; Singh, S. Entity-Based Knowledge Conflicts in Question Answering. In Proceedings of the Proc. EMNLP, 2021. [Google Scholar]
  8. Xie, J.; Zhang, K.; Chen, J.; Lou, R.; Su, Y. Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts. In Proceedings of the Proc. ICLR, 2024. [Google Scholar]
  9. Magesh, V.; Surani, F.; Dahl, M.; Suzgun, M.; Manning, C.D.; Ho, D.E. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. arXiv 2024, arXiv:2405.20362. [Google Scholar]
  10. Wu, K.; Wu, E.; Zou, J. ClashEval: Quantifying the Tug-of-War between an LLM’s Internal Prior and External Evidence. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2024. [Google Scholar]
  11. Shi, F.; Chen, X.; Misra, K.; Scales, N.; Dohan, D.; Chi, E.; Schärli, N.; Zhou, D. Large Language Models Can Be Easily Distracted by Irrelevant Context. In Proceedings of the Proc. Int. Conf. Machine Learning (ICML), 2023. [Google Scholar]
  12. Cuconasu, F.; Trappolini, G.; Siciliano, F.; Filice, S.; Campagnano, C.; Maarek, Y.; Tonellotto, N.; Silvestri, F. The Power of Noise: Redefining Retrieval for RAG Systems. In Proceedings of the Proc. ACM SIGIR Conf. Research and Development in Information Retrieval, 2024. [Google Scholar]
  13. Liu, N.F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; Liang, P. Lost in the Middle: How Language Models Use Long Contexts. Trans. Assoc. Comput. Linguist. 2024, 12, 157–173. [Google Scholar] [CrossRef]
  14. Es, S.; James, J.; Espinosa-Anke, L.; Schockaert, S. RAGAS: Automated Evaluation of Retrieval Augmented Generation. arXiv 2023, arXiv:2309.15217. [Google Scholar]
  15. TruEra. TruLens: Evaluation and Tracking for LLM Applications. GitHub repository. Available online: https://github.com/truera/trulens.
  16. Ru, D.; Qiu, L.; Hu, X.; Zhang, T.; Shi, P.; Chang, S.; Jiayang, C.; Wang, C.; Sun, S.; Li, H.; et al. RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2024. [Google Scholar]
  17. Maynez, J.; Narayan, S.; Bohnet, B.; McDonald, R. On Faithfulness and Factuality in Abstractive Summarization. In Proceedings of the Proc. Annual Meeting of the Association for Computational Linguistics (ACL), 2020. [Google Scholar]
  18. Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y.J.; Madotto, A.; Fung, P. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 2023, 55, 1–38. [Google Scholar] [CrossRef]
  19. Huang, L.; Yu, W.; Ma, W.; Zhong, W.; Feng, Z.; Wang, H.; Chen, Q.; Peng, W.; Feng, X.; Qin, B.; et al. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans. Inf. Syst. 2025, 43, 1–55. [Google Scholar] [CrossRef]
  20. Rashkin, H.; Nikolaev, V.; Lamm, M.; Aroyo, L.; Collins, M.; Das, D.; Petrov, S.; Tomar, G.S.; Turc, I.; Reitter, D. Measuring Attribution in Natural Language Generation Models. Comput. Linguist. 2023, 49. [Google Scholar] [CrossRef]
  21. Min, S.; Krishna, K.; Lyu, X.; Lewis, M.; tau Yih, W.; Koh, P.W.; Iyyer, M.; Zettlemoyer, L.; Hajishirzi, H. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. In Proceedings of the Proc. EMNLP, 2023. [Google Scholar]
  22. Manakul, P.; Liusie, A.; Gales, M.J.F. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. In Proceedings of the Proc. EMNLP, 2023. [Google Scholar]
  23. Gao, T.; Yen, H.; Yu, J.; Chen, D. Enabling Large Language Models to Generate Text with Citations. In Proceedings of the Proc. EMNLP, 2023. [Google Scholar]
  24. Zheng, L.; Chiang, W.L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.P.; et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023. [Google Scholar]
  25. Tang, L.; Laban, P.; Durrett, G. MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents. In Proceedings of the Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. [Google Scholar]
  26. Vectara. HHEM: Hughes Hallucination Evaluation Model. 2023. Available online: https://huggingface.co/vectara/hallucination_evaluation_model.
  27. Saad-Falcon, J.; Khattab, O.; Potts, C.; Zaharia, M. ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems. In Proceedings of the Proc. NAACL, 2024. [Google Scholar]
  28. Niu, C.; Wu, Y.; Zhu, J.; Xu, S.; Shum, K.; Zhong, R.; Song, J.; Zhang, T. RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. In Proceedings of the Proc. ACL, 2024. [Google Scholar]
  29. Zhou, W.; Zhang, S.; Poon, H.; Chen, M. Context-Faithful Prompting for Large Language Models. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP, 2023. [Google Scholar]
  30. Rebedea, T.; Dinu, R.; Sreedhar, M.; Parisien, C.; Cohen, J. NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. In Proceedings of the Proc. EMNLP: System Demonstrations, 2023. [Google Scholar]
  31. Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; Hajishirzi, H. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. In Proceedings of the Int. Conf. Learning Representations (ICLR), 2024. [Google Scholar]
  32. Yoran, O.; Wolfson, T.; Ram, O.; Berant, J. Making Retrieval-Augmented Language Models Robust to Irrelevant Context. In Proceedings of the Proc. ICLR, 2024. [Google Scholar]
  33. Rajpurkar, P.; Jia, R.; Liang, P. Know What You Don’t Know: Unanswerable Questions for SQuAD. In Proceedings of the Proc. ACL, 2018. [Google Scholar]
  34. Nussbaum, Z.; Morris, J.X.; Duderstadt, B.; Mulyar, A. Nomic Embed: Training a Reproducible Long Context Text Embedder, 2025. arXiv arXiv:cs.CL/2402.01613.
  35. Ollama. Ollama: Run Large Language Models Locally. Available online: https://ollama.com.
  36. Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. Qwen3 Technical Report. arXiv 2025, arXiv:cs.CL/2505.09388. [Google Scholar]
  37. Guo, D.; Yang, D.; Zhang, H.; Song, J.; Wang, P.; Zhu, Q.; Xu, R.; Zhang, R.; Ma, S.; Bi, X.; et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 2025, 645, 633–638. [Google Scholar] [CrossRef] [PubMed]
  38. Grattafiori, A.; Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Vaughan, A.; et al. The Llama 3 Herd of Models. arXiv 2024, arXiv:cs.AI/2407.21783. [Google Scholar]
  39. Google. Gemini API Models Documentation. Model gemini-3.8-flash. Available online: https://ai.google.dev/gemini-api/docs/models.
  40. Anthropic. Claude Models Overview. Model claude-haiku-4-5. Available online: https://docs.anthropic.com/en/docs/about-claude/models.
  41. Cohen, J. A Coefficient of Agreement for Nominal Scales. Educ. Psychol. Meas. 1960, 20, 37–46. [Google Scholar] [CrossRef]
  42. McNemar, Q. Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages. Psychometrika 1947, 12, 153–157. [Google Scholar] [CrossRef] [PubMed]
  43. Wilcoxon, F. Individual Comparisons by Ranking Methods. Biom. Bull. 1945, 1, 80–83. [Google Scholar] [CrossRef]
Figure 1. Correctness (left; dashed line: closed-book) and faithfulness of non-refused answers (right), with 95% bootstrap confidence intervals over questions.
Figure 1. Correctness (left; dashed line: closed-book) and faithfulness of non-refused answers (right), with 95% bootstrap confidence intervals over questions.
Preprints 244382 g001
Figure 2. C4 outcomes by mitigation: gold answer only, injected falsehood only, both, neither, or refusal.
Figure 2. C4 outcomes by mitigation: gold answer only, injected falsehood only, both, neither, or refusal.
Preprints 244382 g002
Figure 3. Refusal rates: appropriate (C5, split by question type) and unnecessary (C0–C4).
Figure 3. Refusal rates: appropriate (C5, split by question type) and unnecessary (C0–C4).
Preprints 244382 g003
Table 1. Context conditions (“gold” is the passage containing the answer).
Table 1. Context conditions (“gold” is the passage containing the answer).
Code Condition Passages provided
C0 Clean Gold only
C1 Random distractors Gold + 3 passages from other articles
C2 Related distractors Gold + 3 similar passages without the answer
C3 Heavy noise Gold + 3 related + 3 random
C4 Contradictory Gold + answer-substituted copy + 2 related
C5 Answer absent 4 related passages, gold removed
Table 2. Qwen3-8B results by condition and mitigation ( n = 75 per cell; n = 95 for C5). Correct: lenient match (for C5, refusing). Strict: exact containment. Faithfulness and answer relevance: means over non-refused answers. Context precision is identical across mitigations. Fake: C4 answers matching the injected falsehood. A dash means the measure does not apply.
Table 2. Qwen3-8B results by condition and mitigation ( n = 75 per cell; n = 95 for C5). Correct: lenient match (for C5, refusing). Strict: exact containment. Faithfulness and answer relevance: means over non-refused answers. Context precision is identical across mitigations. Fake: C4 answers matching the injected falsehood. A dash means the measure does not apply.
Cond. Mit. Correct Strict Refusal Faith. Ans. rel. Ctx. prec. Fake Cond. Mit. Correct Strict Refusal Faith. Ans. rel. Ctx. prec. Fake
C0 M0 .947 .920 .000 .940 .916 .987 – C3 M0 .920 .853 .000 .896 .912 .187 –
M1 .933 .880 .027 .927 .911 .987 – M1 .947 .880 .000 .900 .902 .187 –
M2 .947 .947 .013 .905 .914 .987 – M2 .947 .920 .000 .898 .919 .187 –
C1 M0 .907 .880 .000 .909 .920 .247 – C4 M0 .707 .640 .000 .927 .921 .557 .333
M1 .933 .893 .027 .909 .920 .247 – M1 .667 .613 .040 .875 .903 .557 .333
M2 .947 .933 .013 .885 .919 .247 – M2 .587 .560 .000 .893 .920 .557 .413
C2 M0 .920 .867 .000 .916 .914 .327 – C5 M0 .074 .074 .074 .330 .864 .124 –
M1 .907 .840 .013 .919 .895 .327 – M1 .547 .547 .547 .419 .855 .124 –
M2 .920 .893 .000 .891 .922 .327 – M2 .411 .411 .411 .290 .892 .124 –
Closed-book (no context): correct .320, strict .187, refusal .000
Table 3. Faithfulness ( ≥ 0.5 ) vs. correctness, non-refused answers pooled over mitigations (counts). One refused M1 answer in C4 is correct under the lenient rule in Table 2 but is excluded here.
Table 3. Faithfulness ( ≥ 0.5 ) vs. correctness, non-refused answers pooled over mitigations (counts). One refused M1 answer in C4 is correct under the lenient rule in Table 2 but is excluded here.
Faithful Unfaithful
Cond. Correct Wrong Correct Wrong
C0 200 9 12 1
C1 193 10 16 3
C2 193 13 13 5
C3 196 8 15 6
C4 138 65 8 11
Table 4. Rule-based results for the three generators (M0 / M1 / M2). C0 and C4: correctness. Fake: C4 answers adopting the falsehood. No ans.: C4 answers left empty. C5 ref.: appropriate refusals. Unnec.: refusals of answerable C0–C4 questions.
Table 4. Rule-based results for the three generators (M0 / M1 / M2). C0 and C4: correctness. Fake: C4 answers adopting the falsehood. No ans.: C4 answers left empty. C5 ref.: appropriate refusals. Unnec.: refusals of answerable C0–C4 questions.
Model C0 C4 Fake No ans. C5 ref. Unnec.
Qwen3-8B .95/.93/.95 .71/.67/.59 .33/.33/.41 0/0/0 .07/.55/.41 0/.02/.01
R1-32B .95/.95/.99 .75/.67/.72 .23/.28/.32 .04/.11/.16 .26/.61/.67 0/0/.003
Llama-8B .97/.95/.21 .71/.60/.13 .45/.48/.11 0/0/0 .27/.79/.99 0/.09/.79
Table 5. Lenient correctness (%) for the two generators without a per-condition table, under M0 and M1 (75 answerable questions per cell). Max gap is the largest difference from C0; the largest is 8.0 points (Llama, M1, C3).
Table 5. Lenient correctness (%) for the two generators without a per-condition table, under M0 and M1 (75 answerable questions per cell). Max gap is the largest difference from C0; the largest is 8.0 points (Llama, M1, C3).
Model Mit. C0 C1 C2 C3 Max gap
R1-32B M0 94.7 96.0 93.3 93.3 1.3
R1-32B M1 94.7 97.3 94.7 94.7 2.7
Llama-8B M0 97.3 100.0 98.7 98.7 2.7
Llama-8B M1 94.7 94.7 88.0 86.7 8.0
Table 6. First annotator’s support labels by judge verdict (counts).
Table 6. First annotator’s support labels by judge verdict (counts).
Judge verdict Supported Partial Unsupp. Total
Faithful ( ≥ 0.5 ) 32 1 2 35
Unfaithful ( < 0.5 ) 20 6 10 36
Table 7. Judges against the two annotators (71 non-refused Qwen3-8B answers, verdict-balanced sample; faithful = score ≥ 0.5 vs. “supported”), and mean faithfulness of non-refused C4 answers that adopted the falsehood.
Table 7. Judges against the two annotators (71 non-refused Qwen3-8B answers, verdict-balanced sample; faithful = score ≥ 0.5 vs. “supported”), and mean faithfulness of non-refused C4 answers that adopted the falsehood.
Acc. / κ vs. annotator Faith. of fake C4
Judge A B Qwen R1 / Llama
Local Qwen 27B .68 / .36 .66 / .33 .83 .94 / .82
Gemini Flash .73 / .37 .75 / .40 .86 .88 / .87
Claude Haiku .76 / .47 .78 / .50 .62 .82 / .66
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.