Auditable Evaluation Protocol for Spam Detection Under Bounded Paraphrase Attacks

1. Introduction

Email filters must preserve spam recall, limit false positives, and fit local deployment constraints. Large language models can generate fluent malicious text and can support semantic detection controls, but their compute and data-governance demands differ from a local CPU pipeline. We ask what a local detector can support when its attack, corpus, split, evaluation files, and uncertainty analysis are all recorded. Classical spam-filter evaluation and the Enron-Spam benchmark motivate the retained baselines [1,2], while TREC 2007 is retained as a historical spam-track reference [3].

Recent work documents both the ability of large language models to create convincing phishing text and their use as spam-detection controls [4]. Rather than extrapolating external headline metrics, this manuscript reports a local, frozen protocol and names the attack capabilities that were actually tested.

The paper records separate L1, L2, and L3 protocols and denominators; separates Track A in-domain results from Track B transfer and adaptation results; and compares traditional models with MiniLM and DistilBERT embedding-plus-logistic-regression controls on frozen IDs. Its claims follow the frozen predictions, adjudication files, confidence intervals, and paired tests. This artifact-centered framing follows reproducibility and dataset-documentation practice: retain code, data version, environment, and the evaluation record needed to inspect a result [5,6,7,8,9].

Table 1. Evaluated paraphrase threat levels and reporting boundary.

Level	Generator and feedback	Evaluation denominator	Interpretation boundary
L1	Deterministic surface rewriting; no model feedback	All 558 Track-A test spam	Controlled surface-form perturbation
L2	Local generator; non-adaptive	38 human-valid candidates from 100 screened	No target-prediction feedback
L3	Local generator; at most three target queries	55 human-valid candidates from 100 screened	Bounded local adaptive setting; not commercial full restructuring

Table 1. Evaluated paraphrase threat levels and reporting boundary.

Level	Generator and feedback	Evaluation denominator	Interpretation boundary
L1	Deterministic surface rewriting; no model feedback	All 558 Track-A test spam	Controlled surface-form perturbation
L2	Local generator; non-adaptive	38 human-valid candidates from 100 screened	No target-prediction feedback
L3	Local generator; at most three target queries	55 human-valid candidates from 100 screened	Bounded local adaptive setting; not commercial full restructuring

2. Related Work and Research Scope

2.1. Learning-Based Email Filtering

Learning-based spam filtering has long been framed as text categorization with asymmetric operational costs: a false positive can suppress legitimate mail, whereas a false negative exposes a user to unwanted or harmful content. Early Naive Bayes evaluations established corpus-specific training and evaluation protocols [1], while later work compared alternative Bayes formulations and preprocessing choices [2]. Text categorization research also established sparse high-dimensional learning as a practical setting for support-vector methods [10]. Recent phishing research shows that language models can remove superficial writing cues and support iterative, context-dependent message generation [11,12]. At the same time, recent benchmark studies warn that unrepresentative benchmark populations, duplicate overlaps, and opaque model exposure can inflate apparent results [13,14]. Support-vector filtering demonstrated that sparse textual representations could give competitive performance and practical training time [15]. These studies motivate the retained classical baselines, but they do not remove the need to audit duplicate messages, temporal or source overlap, and metric denominators in current reproductions.

A classifier score is only one part of the result. The cohort, split manifest, text normalization, fitted vocabulary, calibration partition, attack generator, and paired prediction vector determine what that score means. Without them, an accuracy difference can reflect a different evaluation population. This is common in public mail collections assembled from multiple folders or republished in curated form.

This protocol-oriented position is related to established work on online supervised spam-filter evaluation, which formalized the need to specify the evaluation population, contamination controls, and metric denominators for a spam-filter score [16]. It differs in retaining both attack-side review decisions and paired prediction artifacts. Deduplication studies show that overlap between training and test data can degrade learned representations [17]; dataset-documentation and contamination-tracing work shows that a reported score needs a stated population, a stated overlap policy, and a stated exposure boundary [6,7,8]. A recent re-evaluation of phishing-email benchmarks finds that near-perfect in-dataset results can conceal limited realism and motivates transparent reimplementation and dataset documentation [14]. The present work therefore treats exact and group-near deduplication as explicitly different cohort policies rather than interchangeable preprocessing details.

2.2. Paraphrase Attacks and Language-Model Controls

Natural-language adversarial evaluation differs from image perturbation evaluation because candidate validity cannot be inferred from edit distance alone. Word substitutions may change an entity, add a harmful promise, reverse an instruction, or yield unreadable text. NLP attack studies based on substitutions, semantic constraints, and saliency [18,19,20,21,22] motivate explicit candidate-validity controls. Recent phishing work further shows critique-refinement generation [12] and multi-round fraud inducements [23], reinforcing the need to distinguish fluent output from a comparable, valid attack sample. In the email setting, language models can produce fluent phishing material [4,24], while recent benchmark work evaluates conventional and language-model detectors under a common phishing-email protocol [14]. Those findings justify testing a bounded local generator, but they do not justify claims about a different commercial model, an unrestricted attack budget, or a message with changed operational intent.

The modern controls use fixed all-MiniLM-L6-v2 or DistilBERT-base-uncased embeddings followed by logistic regression. Transformer self-attention provides the architectural basis [25], BERT provides the contextual pretraining basis [26], Sentence-BERT provides the embedding formulation [27], and DistilBERT and MiniLM document compression routes for Transformer representations [28,29]. These controls test representation transfer on the same frozen manifests. They are not fully fine-tuned email detectors.

Table 2. Scope of related approaches and their role in this study.

Approach	What it contributes	What is not inferred here	Study role
Classical sparse text filters	Interpretable local baseline	Universal robustness	NB, SVM, RF and RMF-PC
Embedding plus linear controls	Fixed contextual representation	Fine-tuned Transformer ranking	MiniLM and DistilBERT controls
Rule surface rewrite	Deterministic rule-defined perturbation; Appendix A2	Semantic preservation by itself	L1 test
Local LLM paraphrase	Fluent candidate generation	Commercial or unlimited attacks	L2/L3 after human review

Table 2. Scope of related approaches and their role in this study.

Approach	What it contributes	What is not inferred here	Study role
Classical sparse text filters	Interpretable local baseline	Universal robustness	NB, SVM, RF and RMF-PC
Embedding plus linear controls	Fixed contextual representation	Fine-tuned Transformer ranking	MiniLM and DistilBERT controls
Rule surface rewrite	Deterministic rule-defined perturbation; Appendix A2	Semantic preservation by itself	L1 test
Local LLM paraphrase	Fluent candidate generation	Commercial or unlimited attacks	L2/L3 after human review

2.3. Research Questions and Evidence Boundary

The study is organized around four questions. RQ1 asks whether a sparse local detector maintains useful L1 spam recall on an exact-deduplicated in-domain split. RQ2 asks whether the word, character, fusion, augmentation, and gate components change that result under the same attack list. RQ3 asks whether the observed behavior persists when the source-trained models are evaluated on separately prepared Enron and curated TREC targets and when a small, isolated target-domain label budget is introduced. RQ4 asks whether a human-validated local LLM paraphrase set changes the empirical miss rate within an explicitly bounded attack budget. Each question has a separate denominator and cannot be pooled into a single universal robustness score.

Table 3. Evidence map and non-claims.

Question	Primary evidence	Supported statement	Explicit non-claim
RQ1	Track-A frozen clean/L1 vectors	Protocol-specific in-domain FNR	Robustness to all paraphrases
RQ2	Component ablations and paired tests	Observed component differences	Statistical superiority when interval crosses zero
RQ3	Track-B targets and isolated adaptation splits	Distribution shift and limited adaptation behavior	Official raw TREC performance or general domain invariance
RQ4	Review and adjudication records	Human-validated bounded local attack outcome	Success rate for unreviewed candidates or unrestricted LLMs

Table 3. Evidence map and non-claims.

Question	Primary evidence	Supported statement	Explicit non-claim
RQ1	Track-A frozen clean/L1 vectors	Protocol-specific in-domain FNR	Robustness to all paraphrases
RQ2	Component ablations and paired tests	Observed component differences	Statistical superiority when interval crosses zero
RQ3	Track-B targets and isolated adaptation splits	Distribution shift and limited adaptation behavior	Official raw TREC performance or general domain invariance
RQ4	Review and adjudication records	Human-validated bounded local attack outcome	Success rate for unreviewed candidates or unrestricted LLMs

3. RMF-PC Method and Design Rationale

3.1. RMF-PC and Deployment Boundary

RMF-PC operates on subject and body text. Its word view is 1-2 gram TF-IDF (English stop words removed, min_df=2, sublinear term frequency) and its character view is char_wb 2-5 gram TF-IDF concatenated with seven standardized structural features: length, word count, URL count, phone-number count, digit ratio, uppercase ratio, and punctuation count. The structural features are therefore part of the character-plus-structural logistic-regression branch, not a third separately weighted fusion input. The word classifier is a linear SVM with hinge-loss labels t=2y-1; when its score is fused with probabilistic outputs, calibration is fitted only within the training or validation partitions. The optional gate is activated only near the probability decision threshold, creates three rewrite copies, and aggregates their predictions. It is an optional defense-in-depth mechanism rather than a universal improvement.

The linear SVM and random-forest baselines follow standard formulations [30,31], and all TF-IDF, calibration, classification, and metric calculations use the documented scikit-learn implementation [32].

Figure 1. Core RMF-PC architecture and evidence boundary. Fitting, calibration and fusion are confined to training-side partitions; the consistency gate is optional and logged separately.

Figure 1. Core RMF-PC architecture and evidence boundary. Fitting, calibration and fusion are confined to training-side partitions; the consistency gate is optional and logged separately.

3.2. Fitting Order and Decision Rule

All data-dependent transformations are fitted after the split. For each model, the training manifest fixes the message IDs and labels; the vectorizer vocabulary and inverse-document-frequency statistics are fitted on training text only. Where a calibration or fusion rule is needed, its inputs are generated from a training-side validation partition rather than the held-out evaluation files. Augmentation is applied only to training documents, and attack candidates are generated only after the evaluation samples have been frozen. This order prevents an attacked test message, a test-domain token frequency, or a test probability from influencing the fitted representation or threshold; it also avoids the leakage and evaluation malpractices documented for repeatedly reused or opaque evaluation data [33].

Let x_w denote the word TF-IDF vector and let x_cs denote the concatenation of character TF-IDF and standardized structural features. The word classifier uses a signed linear decision function f_w(x_w), with the hinge-loss label convention t=2y-1. Training-side calibration produces p_w, while a logistic-regression classifier produces p_cs. The fused probability is p=0.45p_w+0.55p_cs; there is no unreported third structural weight. The gate is invoked when |p-0.5|<0.20, i.e., p lies strictly between 0.30 and 0.70. It creates three deterministic local surface variants, predicts them with the frozen classifier, and takes the median of the original and three variant probabilities. The retained experiment did not log the natural test-traffic gate-trigger prevalence; accordingly, no amortized gate-cost or operational value claim is made.

Algorithm 1 gives the experimental order rather than a claim that the procedure is optimal. It provides the minimum sequence required for the reported result to be reproducible. The step that consumes the attack generator is deliberately downstream of model fitting and test-manifest freezing.

Algorithm 1. Leakage-controlled RMF-PC evaluation procedure.

Step	Operation	Artifact retained
1	Audit archives; remove duplicates according to the declared track policy	Archive hashes and retained IDs
2	Create stratified seed-42 source split before any augmentation	Train/test manifests
3	Fit vectorizers, classifiers, calibrators and fusion only on training-side data	Serialized configuration and versions
4	Freeze clean predictions; generate L1, L2 or L3 candidates from test IDs	Prediction and attack manifests
5	Review L2/L3 candidate validity before formal denominators are formed	Reviewer forms and adjudication
6	Compute paired metrics and intervals from frozen vectors	Metric, bootstrap and test outputs

3.3. Frozen Implementation Specification

The frozen run uses word TF-IDF with 1-2 grams, English stop-word removal, min_df=2 and sublinear term frequency; character TF-IDF uses char_wb 2-5 grams, min_df=2, sublinear term frequency, and at most 100,000 features. The character-plus-structural branch is LogisticRegression(max_iter=1000, random_state=42). The word SVM is CalibratedClassifierCV(LinearSVC(random_state=42), cv=3, method=“sigmoid”); scikit-learn defaults therefore set C=1.0 and class_weight=None. RMF-PC uses p=0.55p_cs+0.45p_w, ambiguity band delta=0.20, and k=3 gate variants. Training augmentation appends two L1 variants to each training message. Baselines are default Naive Bayes, LinearSVC (random state 42), and RandomForestClassifier with 200 trees, random state 42, and one worker. These values describe the retained implementation; they were not tuned on the test set.

L1 is a deterministic rule-defined surface transformation. It preserves URL, email-address, currency, and numeric spans; independently applies synonym substitutions with probability 0.85, single-letter u/r expansions with probability 0.70, reversal of one contiguous four-word span only for segments of at least 12 words with probability 0.60, a greeting prefix with probability 0.85, and a closing line with probability 0.70. The complete synonym mapping and protected-token expression are listed in Appendix A2. Each training message receives two seeded L1 variants; each test attack receives one deterministic seeded variant. L1 is not a semantic-preservation proof. L2/L3 decoding fields retained in the frozen record are seed 42, temperature 0.7, top-p 0.9, maximum 192 tokens, and the L3 maximum of three target queries. A human-readable generator release label, full prompt text, and complete candidate-selection policy were not retained; the manuscript therefore does not make generator-specific or yield-wide claims.

3.4. Complexity and Practical Deployment Implications

For sparse linear text models, prediction scales approximately with the nonzero feature count, while the random-forest cost additionally depends on the number and depth of trees. The character view offers tolerance to superficial token changes but can increase sparsity and feature extraction cost. The gate triggers extra preprocessing and prediction only for threshold-adjacent messages. Because a natural test-traffic trigger prevalence was not retained, the present work does not estimate its amortized service cost or make a lightweight-latency claim.

The evaluation distinguishes local inference, CPU feasibility, latency, and robustness. A strict service-level target is not claimed. The retained timing evidence supports only the stated same-machine comparison between the two modern controls; it does not support a numerical latency ranking against RMF-PC.

4. Experimental Protocol and Reproducibility

4.1. Data Tracks, Provenance, and Leakage Controls

Track A used 10,751 raw Apache SpamAssassin messages obtained from nine public-corpus archives [34]: 6,954 ham and 3,797 spam. The category composition was easy_ham 5,052, easy_ham_2 1,401, hard_ham 501, spam 1,002, and spam_2 2,795. One message yielded empty extracted text and was excluded before exact-content auditing; no unreadable files were recorded. The remaining 10,750 messages contained 4,225 duplicate groups (4,570 duplicate messages beyond the first) and no cross-label duplicate group. Exact duplicate removal therefore retained 6,180 messages; the stratified seed-42 manifest contains 3,025 ham / 1,301 spam in training and 1,296 ham / 558 spam in test. Archive-level SHA-256 values are retained in Appendix A, along with names and counts.

Track B is a separate group-near-deduplicated source cohort: 6,170 messages split into 4,319 source training and 1,851 source test messages (1,295 ham; 556 spam). It supports transfer and adaptation only and is never pooled with Track A. Enron retained 29,921 target messages (15,590 ham; 14,331 spam) after cross-source screening. The TREC target contains 53,459 normalized messages (24,344 ham; 29,115 spam) from the third-party Zenodo TREC_07.csv curated version (record 8339691), not an official NIST raw-corpus download; all TREC claims are limited to that frozen curated input [35,36]. Public corpus provenance does not demonstrate that a benchmark was absent from the pretraining data of every language model, so the manuscript does not make that claim [37]. The retained target labels were not independently reconciled against an official raw-corpus mapping. No raw email body or attachment is redistributed with the reproducibility package. Source-specific license or access terms govern downstream retrieval and redistribution; where no SPDX license field was retained in the local audit, the manuscript does not infer one.

Exact deduplication identifies byte- or normalized-content-identical messages within the Track-A corpus. Group-near deduplication is a stricter cross-source policy used for Track B to suppress highly similar messages before transfer evaluation. These policies create different retained cohorts and are therefore reported as separate tracks rather than alternative preprocessing of a shared denominator. File presence, category assignment, archive hash, duplicate counts, class proportions, and split IDs were retained in dataset-audit and manifest outputs. No target-domain test message was used to choose a source-trained hyperparameter.

Table 4. Data and evaluation isolation.

Track	Source / target	Split or target size	Use
A	SpamAssassin, exact deduplication	4,326 train / 1,854 test; 558 test spam	In-domain L1, ablation, L2/L3
B	SpamAssassin, group-near deduplication	4,319 source train / 1,851 source test	Source-trained transfer and adaptation
B target	Enron after cross-source screening	29,921: 15,590 ham / 14,331 spam	Transfer and independent adaptation test
B target	TREC 2007 Zenodo curated version	53,459: 24,344 ham / 29,115 spam	Transfer and independent adaptation test

Table 4. Data and evaluation isolation.

Track	Source / target	Split or target size	Use
A	SpamAssassin, exact deduplication	4,326 train / 1,854 test; 558 test spam	In-domain L1, ablation, L2/L3
B	SpamAssassin, group-near deduplication	4,319 source train / 1,851 source test	Source-trained transfer and adaptation
B target	Enron after cross-source screening	29,921: 15,590 ham / 14,331 spam	Transfer and independent adaptation test
B target	TREC 2007 Zenodo curated version	53,459: 24,344 ham / 29,115 spam	Transfer and independent adaptation test

Figure 2. Experimental evidence and data-isolation protocol. One Track-A message with empty extracted text is excluded before the exact-deduplication audit. Track A and Track B use separate preprocessing cohorts and are never pooled.

Figure 2. Experimental evidence and data-isolation protocol. One Track-A message with empty extracted text is excluded before the exact-deduplication audit. Track A and Track B use separate preprocessing cohorts and are never pooled.

Nine frozen prediction pipelines are compared: Naive Bayes, linear SVM, random forest, RMF word, RMF character, RMF fusion, RMF gated fusion, all-MiniLM-L6-v2 embeddings plus logistic regression, and DistilBERT-base-uncased embeddings plus logistic regression. The transformer controls are offline embedding-plus-linear models, not transformer fine-tuning. Sentence-BERT style embedding models and distilled Transformers motivate these controls [27,28].

4.2. Threat Model and Candidate-Validity Review

The capability framework separates the threat from the test artifact. C1 is a surface-form capability: spelling, punctuation, whitespace, or limited synonym changes. C2 is a semantically constrained paraphrase capability that aims to preserve the message proposition. C3 adds bounded query feedback to a local target model. C4 would allow unrestricted semantic restructuring, multi-turn planning, external tools, or a commercial LLM; it is out of scope. L1 operationalizes C1 using deterministic rules; L2 operationalizes a fixed local model without target feedback; L3 operationalizes a fixed local model with at most three target queries. No C4 result is reported.

For L2 and L3, a candidate enters the formal denominator only after independent assessment of four criteria: semantic preservation, named-entity preservation, absence of newly introduced harmful content, and fluency. A failed criterion means the candidate may remain in a screening log but cannot count as a successful evasion or a tested attack. Adjudication resolves reviewer disagreement. This is a conservative choice: it reduces the effective sample size but avoids reporting a prediction change for a meaning-changed email as a paraphrase evasion.

Table 5. Human-review rubric for L2/L3 candidate eligibility.

Criterion	Pass condition	Examples requiring exclusion
Semantic preservation	Core request, offer and intended action are unchanged	A payment request becomes a meeting invitation
Entity preservation	People, organizations, amounts, URLs and dates are retained or equivalently masked	Replacing a named bank, amount, or destination URL
No harm addition	No new coercion, credential request, malware claim or financial promise	Adding a threat or a new credential-harvesting step
Fluency	Message is comprehensible English without obvious corruption	Broken fragments or nonsensical substitutions

Table 5. Human-review rubric for L2/L3 candidate eligibility.

Criterion	Pass condition	Examples requiring exclusion
Semantic preservation	Core request, offer and intended action are unchanged	A payment request becomes a meeting invitation
Entity preservation	People, organizations, amounts, URLs and dates are retained or equivalently masked	Replacing a named bank, amount, or destination URL
No harm addition	No new coercion, credential request, malware claim or financial promise	Adding a threat or a new credential-harvesting step
Fluency	Message is comprehensible English without obvious corruption	Broken fragments or nonsensical substitutions

4.3. Metrics and Statistical Analysis

Accuracy, precision, recall, F1, false-positive rate (FPR), and spam miss rate FNR=FN/(TP+FN) are calculated from frozen prediction vectors. FPR and ROC-oriented diagnostic thinking follow standard classifier-evaluation practice [38]. L1 FNR uses every test-spam message. L2/L3 all-effective-candidate miss rate uses only human-adjudicated effective candidates; conditional attack success is calculated only among candidates whose originals were correctly detected. Wilson intervals are used for binomial FNR [39]. Pairwise comparisons use 10,000 matched bootstrap replicates [40] and two-sided exact McNemar tests [41]; Track-B family-wise p values are Holm-adjusted [42].

Wilson intervals quantify uncertainty in a binomial miss rate. Matched bootstrap intervals quantify the difference between two models on the same messages. McNemar tests discordant paired decisions. They answer different questions, and a non-significant paired result does not make two models equivalent. Calibration is kept separate from evaluation because score-to-probability fitting must not use test outcomes [43,44,45]. We use paired, per-corpus inference rather than pooling corpora or comparing independently computed summaries [46] and report the confusion-matrix measures described by Sokolova and Lapalme [47]. The analysis output retains Holm-adjusted p values for Track-B comparisons; the F1/FNR tables remain descriptive because corpus shift rules out a single global ranking.

4.4. Execution Environment and Reproducibility Record

The fixed random seed is 42. The reproducibility record captures Python and library versions, operating-system and CPU information, archive SHA-256 values, commands, input manifests, model configuration, output hashes, and elapsed time. The modern-control timing uses the Ubuntu VM stated in Section 5.4. No cross-model latency comparison is made because no same-machine RMF-PC latency record is retained.

Table 6. Reproducibility artifacts retained with the experiment.

Artifact	Purpose	Use in audit
Dataset audit JSON and Markdown	Archive identity, extraction and duplicate policy	Verify input provenance
Split and prediction manifests	Stable IDs, labels and frozen outputs	Recompute paired metrics
Attack trajectory and candidate files	L1 rule output and L2/L3 generation trace	Separate screening from formal samples
Reviewer and adjudication CSVs	Eligibility decisions and reasons	Freeze L2/L3 denominator
Environment and execution logs	Versions, CPU, commands and runtime	Repeat implementation path

Table 6. Reproducibility artifacts retained with the experiment.

Artifact	Purpose	Use in audit
Dataset audit JSON and Markdown	Archive identity, extraction and duplicate policy	Verify input provenance
Split and prediction manifests	Stable IDs, labels and frozen outputs	Recompute paired metrics
Attack trajectory and candidate files	L1 rule output and L2/L3 generation trace	Separate screening from formal samples
Reviewer and adjudication CSVs	Eligibility decisions and reasons	Freeze L2/L3 denominator
Environment and execution logs	Versions, CPU, commands and runtime	Repeat implementation path

5. Results

5.1. Track A Fixed-Split L1 Results

Gated RMF-PC produced a 1.971% L1 FNR (11/558), compared with 2.509% (14/558) for linear SVM. The paired comparison has only nine discordant decisions and does not establish superiority: exact McNemar p=0.5078125 and the paired-bootstrap FNR-difference interval crosses zero. It should not be read as equivalence either: no equivalence margin or minimum detectable effect was prespecified. F1 is shown alongside FNR and FPR because an attacked F1 can increase while false-positive behavior changes.

On the same frozen Track-A IDs, DistilBERT plus logistic regression had a lower L1 FNR than gated RMF-PC (0.896%, 5/558 versus 1.971%, 11/558), whereas gated RMF-PC had the higher L1 F1 (98.117% versus 96.763%). The paired gated-minus-DistilBERT FNR difference is +1.075 percentage points (95% bootstrap interval 0.000 to 2.330; exact McNemar p=0.1460). These results are reported as a trade-off on this cohort, not as a universal ranking.

Table 7. Track-A fixed-split L1 comparison (seed 42; 1,296 ham and 558 spam).

Model	Clean F1%	L1 F1%	L1 FPR %	L1 spam FNR %
Naive Bayes	88.801	87.800	0.231	21.326
Linear SVM	96.804	98.018	0.617	2.509
Random forest	96.596	96.310	0.309	6.452
RMF-PC word	96.751	97.948	1.080	1.613
RMF-PC char	97.080	97.464	0.617	3.584
RMF-PC fusion	97.838	98.025	0.772	2.151
RMF-PC gated fusion	97.937	98.117	0.772	1.971
DistilBERT embedding + LR	97.948	96.763	2.469	0.896

Table 7. Track-A fixed-split L1 comparison (seed 42; 1,296 ham and 558 spam).

Model	Clean F1%	L1 F1%	L1 FPR %	L1 spam FNR %
Naive Bayes	88.801	87.800	0.231	21.326
Linear SVM	96.804	98.018	0.617	2.509
Random forest	96.596	96.310	0.309	6.452
RMF-PC word	96.751	97.948	1.080	1.613
RMF-PC char	97.080	97.464	0.617	3.584
RMF-PC fusion	97.838	98.025	0.772	2.151
RMF-PC gated fusion	97.937	98.117	0.772	1.971
DistilBERT embedding + LR	97.948	96.763	2.469	0.896

Audit note: DistilBERT Clean and RMF-PC word L1 have the same aggregate confusion counts (TN=1,282, FP=14, FN=9, TP=549), and therefore the same rounded six metrics. They are not copied predictions: the retained vectors differ on 32 of 1,854 IDs. Table D0 reports the complete independently recomputed rows.

Table 8. Track-A RMF-PC ablations under the same L1 attack.

Configuration	Clean F1%	L1 F1%	L1 FNR %	Interpretation
Word only	96.751	97.948	1.613	Lowest observed component FNR
Char only	97.080	97.464	3.584	Higher FNR than word view
Fusion	97.838	98.025	2.151	Optional combination
Gated fusion	97.937	98.117	1.971	Small observed difference; no superiority claim
Fusion without augmentation	97.561	97.926	2.688	+0.537 pp vs augmented fusion
Gated fusion without augmentation	98.018	97.930	2.509	+0.538 pp vs augmented gated fusion

Table 8. Track-A RMF-PC ablations under the same L1 attack.

Configuration	Clean F1%	L1 F1%	L1 FNR %	Interpretation
Word only	96.751	97.948	1.613	Lowest observed component FNR
Char only	97.080	97.464	3.584	Higher FNR than word view
Fusion	97.838	98.025	2.151	Optional combination
Gated fusion	97.937	98.117	1.971	Small observed difference; no superiority claim
Fusion without augmentation	97.561	97.926	2.688	+0.537 pp vs augmented fusion
Gated fusion without augmentation	98.018	97.930	2.509	+0.538 pp vs augmented gated fusion

The equality between selected values in Table 8 and the linear-SVM L1 row of Table 7 is not a copied entry. The gated no-augmentation and linear-SVM L1 vectors share the same spam miss count (14/558), but differ on 13 of 1,854 IDs; on spam, the paired discordances are n01/n10=5/5 (exact McNemar p=1.000). The no-augmentation metrics are independently derived from the retained no-augmentation prediction bundle. The configurations nevertheless differ, and Table 8 should be interpreted only as the stated ablation comparison.

5.2. Human-Adjudicated Local LLM Attacks

Automatic screening was not treated as a formal attack-success rate. Two reviewers independently assessed semantic preservation, entity preservation, absence of added harm, and fluency; disagreements were adjudicated. From 100 selected candidates per protocol, 38 L2 and 55 L3 candidates were retained as effective. Every retained original was correctly detected, so conditional success equals the reported miss count over the effective denominator. The record does not retain the full generator yield or selection rule before these 100 selected candidates; the results are therefore limited to the adjudicated candidate sets.

Table 9. Human-adjudicated local-generator L2/L3 feasibility observations.

Protocol	Selected / effective	Misses	All-effective miss rate	Conditional success	Boundary
L2 non-adaptive	100 / 38	1	2.632% (Wilson 0.466–13.495%)	1 / 38	No target feedback
L3 bounded adaptive	100 / 55	0	0.000% (Wilson upper 6.529%)	0 / 55	At most 3 queries

Table 9. Human-adjudicated local-generator L2/L3 feasibility observations.

Protocol	Selected / effective	Misses	All-effective miss rate	Conditional success	Boundary
L2 non-adaptive	100 / 38	1	2.632% (Wilson 0.466–13.495%)	1 / 38	No target feedback
L3 bounded adaptive	100 / 55	0	0.000% (Wilson upper 6.529%)	0 / 55	At most 3 queries

The frozen generator summaries identify the local generator by SHA-256 8ffcf166d3467df1c38991fa6196c8798b7e2bb9101d8fa34f6cd3c186913f14. A human-readable generator release label was not retained in the frozen summary, so it is not retroactively inferred here. L2 and L3 parameters and agreement figures are reported in Appendix B; the pre-human L3 automated evasion count remains a screening field and is not used in Table 9.

Table 9. a. Retained L2/L3 generation and review metadata.

Field.	L2 non-adaptive	L3 bounded adaptive
Seed / selected candidates	42 / 100	42 / 100
Threat feedback	None	Frozen gated RMF-PC; max. 3 queries
Temperature / top-p / max tokens	0.7 / 0.9 / 192	0.7 / 0.9 / 192
Threads / context	8 / not recorded	8 / 2,048
Generator yield / selection policy	Not retained	Not retained
Model identifier	SHA-256 retained; release label absent	SHA-256 retained; release label absent

Table 9. a. Retained L2/L3 generation and review metadata.

Field.	L2 non-adaptive	L3 bounded adaptive
Seed / selected candidates	42 / 100	42 / 100
Threat feedback	None	Frozen gated RMF-PC; max. 3 queries
Temperature / top-p / max tokens	0.7 / 0.9 / 192	0.7 / 0.9 / 192
Threads / context	8 / not recorded	8 / 2,048
Generator yield / selection policy	Not retained	Not retained
Model identifier	SHA-256 retained; release label absent	SHA-256 retained; release label absent

5.3. Track B Cross-Corpus Transfer and Adaptation

Track-B values are descriptive source-trained transfer results on frozen target IDs. Table 10 and Table 11 and Appendix D2–D3 are full frozen target-corpus transfer evaluations: Enron covers 29,921 messages (15,590 ham and 14,331 spam), and TREC covers 53,459 messages (24,344 ham and 29,115 spam). They show substantial domain sensitivity and should not be interpreted as a universal corpus ranking (Figure 3). Section 6.5 discusses the protocol sensitivity of the Track-B source-holdout comparison. These full-target transfer cohorts are distinct from the target-domain adaptation models, which use a 30% independent target test set; 5% and 10% label budgets are drawn only from a disjoint 70% adaptation pool. Enron adaptation test contains 4,677 ham and 4,300 spam (n=8,977); TREC adaptation test contains 7,303 ham and 8,735 spam (n=16,038). The retained audit does not preserve the per-class counts sampled into each 5%/10% adaptation subset, so no stronger class-balance claim is made.

Table 12 is evaluated on a different population from the Enron and TREC columns of Table 10 and Table 11: the adaptation models are tested on the independent 30% target test set, whereas the source-trained transfer rows in Table 10 and Table 11 are evaluated on the full target corpus. The two are therefore not directly comparable and must not be differenced.

A model can score well on the SpamAssassin source and lose F1 and spam recall on a target collection. Figure 3 shows the consequence for model selection: RMF-PC char degrades from 6.47% to 72.51% spam FNR on Enron, while DistilBERT + LR also degrades, from 4.50% to 22.96% on Enron and 24.94% on TREC, and is nonetheless the best target-corpus detector of the nine. A source holdout therefore selects a different detector than the target corpora do. Neither L1 rewriting nor an embedding representation removes the need to measure the target distribution. The tested 5% and 10% label budgets improved outcomes on their fixed pools and independent target tests; they do not define a universal labeling requirement.

5.4. Same-CPU Modern-Control Latency

MiniLM and DistilBERT controls were measured on the same Ubuntu VM (Intel Xeon E7-4830 v3 at 2.10 GHz; 15 GiB RAM; Python 3.12.3; PyTorch 2.14.1+cpu). One thread, batch size one, ten warm-up items, 100 fixed IDs, and five repeats produced 500 timed items per control. These values compare the two modern controls only; no same-machine RMF-PC timing exists.

Table 13. Same-CPU modern embedding-control latency (ms; 500 timed items each).

Pipeline	Feature mean	Predict mean	E2E mean	Median	P95	P99	Max	Items/s
MiniLM + LR	428.286	0.667	428.953	483.963	512.894	526.829	668.683	2.331
DistilBERT + LR	1397.511	0.793	1398.304	1584.746	1620.046	1650.847	1748.803	0.715

Table 13. Same-CPU modern embedding-control latency (ms; 500 timed items each).

Pipeline	Feature mean	Predict mean	E2E mean	Median	P95	P99	Max	Items/s
MiniLM + LR	428.286	0.667	428.953	483.963	512.894	526.829	668.683	2.331
DistilBERT + LR	1397.511	0.793	1398.304	1584.746	1620.046	1650.847	1748.803	0.715

The preserved timing summary does not retain per-item minima or quartile counts, so these values are not reconstructed. Mean, median, upper quantiles, and maxima are reported exactly as retained. In both controls, the mean is lower than the median, indicating faster observations in the retained timed set; without per-item timings, the lower tail cannot be characterized further.

6. Discussion

6.1. Interpretation of the In-Domain Result

Track A is a controlled comparison. Word-only RMF-PC had the lowest observed L1 FNR among the listed RMF components, while gated fusion had the highest L1 F1. The difference between gated fusion and linear SVM did not meet the paired-test criterion. Rounded F1 alone would hide FNR, false-positive behavior, paired discordance, and latency.

Augmentation reduced the observed L1 FNR for fusion and gated fusion relative to the listed no-augmentation forms. Because training augmentation and the L1 test derive from the same deterministic rule family, this is a within-family robustness measurement: it shows the effect of exposure to that family, not robustness to unseen transformations, unrestricted paraphrases, or semantic attacks.

The L1 condition is a fixed deterministic perturbation, not a monotone difficulty increase. Table D0 shows a lower observed FNR for six of the eight pipelines, including two baselines that received no L1 augmentation (linear SVM: 28 to 14 of 558; DistilBERT: 9 to 5); it rises for Naive Bayes and random forest. The rule family combines potentially opposing changes: synonym substitution weakens some lexical spam markers, while independently sampled prefix and closing insertion add repeated boilerplate to many messages. These changes affect lexical, character, and embedding representations differently; for example, DistilBERT false positives rise from 14 to 32 while its spam misses fall from 9 to 5. Thus the L1 miss rate is a measurement under one specified perturbation family, not an ordering of attack strength. The within-family ablation difference associated with augmentation is consequently not the only contributor to the observed clean-to-L1 changes.

6.2. Human Review as a Measurement Control

L2/L3 review controls the measurement. A local generator can produce fluent text that changes a bank name, monetary amount, action request, or harm level. Counting it as evasion would mix changed message content with a detection failure. The formal L2/L3 denominators follow four-criterion review and adjudication. We retain selected totals to show the reviewed cohort, and use Wilson intervals because the effective totals are small. These L2/L3 results are a protocol demonstration, not a generator-wide security estimate.

L3 had zero misses among 55 effective candidates, but its two-sided Wilson upper bound is 6.529%. The result cannot support a zero-risk claim or resistance to a general adaptive LLM adversary. A larger study should preregister the prompt and query budget, retain complete trajectories, and use at least two independent reviewers with a documented adjudication rule.

6.3. Cross-Corpus Behavior and Model Selection

The transfer tables show domain shift. Source-trained sparse and embedding controls have materially different FNR values on Enron and the curated TREC target. A practitioner should treat a source-trained score as a screening baseline, audit target-language and source overlap, and reserve an isolated target test set before accepting an adapted classifier. The measured 5% and 10% configurations improved the fixed-split results; they do not specify a universal labeling policy.

MiniLM and DistilBERT serve a second role: they prevent an historical-only comparison. Their transfer behavior is reported alongside the traditional pipelines and their modern-control latency is measured on the same machine. The CPU measurements demonstrate a large feature-extraction cost for these frozen embedding pipelines relative to their logistic-regression head. They are not a cross-model latency comparison, because no same-machine RMF-PC latency measurement is retained.

6.4. Evidence Synthesis Across the Evaluation Layers

The experiment layers are complementary rather than interchangeable. L1 supplies a full, deterministic denominator for an auditable surface-form stress test. L2 and L3 contribute fluent local-generator candidates, but only after validity review reduces the sample to messages that remain comparable to their originals. Cross-corpus transfer measures a different question: whether a source-trained decision rule travels to a different collection. Target-domain adaptation measures whether a bounded amount of new-domain labeling changes an independently held-out target result. A conclusion can be strong within one layer while remaining untested in another.

This separation avoids two common aggregation errors. First, it would be invalid to average Track-A and Track-B FNR values because their duplicate policy, population, and evaluation purpose differ. Second, it would be invalid to replace the human-reviewed L2/L3 denominator with the 100 screened candidates merely because that produces a simpler percentage. The report instead uses the smallest denominator that matches the claim and exposes both the screened and effective counts.

6.5. Cohort Policy as an Experimental Variable

The two SpamAssassin tracks provide a direct protocol-sensitivity case. Under the same gated-versus-linear-SVM L1 comparison, Track A (exact deduplication; 558 test spam) gives an FNR difference of -0.538 percentage points, exact McNemar p=0.5078, and bootstrap interval [-1.613, +0.538]. Track B source holdout (group-near deduplication; 556 spam) gives -1.439 percentage points, raw exact McNemar p=0.0215, and bootstrap interval [-2.518, -0.360]. The cohorts are similar in size but differ in construction; these results show that a deduplication and cohort policy can change an inferential conclusion even when headline FNRs are close. This cohort sensitivity is consistent with, and complements, recent evidence that phishing-email benchmarks can sustain near-perfect in-dataset results whose realism does not transfer [14]. It is not evidence that one policy is universally preferable.

For Track B, the raw p value is below 0.05 and the bootstrap interval excludes zero, but the Holm-adjusted p value across the pre-specified Track-B comparison family is 0.0859. This manuscript treats the Holm-adjusted p value as the family-wise significance decision and reports the bootstrap interval as an effect-estimate uncertainty measure. Thus the Track-B result is described as a protocol-sensitive effect with a negative interval, not as an unqualified family-wise significant superiority claim. Track A remains unresolved under both summaries.

Figure 4 shows the corresponding paired differences and intervals for every retained comparator. The highlighted comparison is the one analysed in this section. Of the seven comparisons available in both cohorts, two change their relation to zero: gated RMF-PC versus linear SVM (Track A [-1.613, +0.538] includes zero; Track B [-2.518, -0.360] excludes it) and gated RMF-PC versus DistilBERT + LR (Track A [+0.000, +2.330] includes zero; Track B [-3.777, -0.180] excludes it). Intervals with a printed bound of +0.000 are treated as including zero. That the same comparator can fall on either side of zero depending on the cohort policy is why this section reports a protocol-sensitivity case rather than a detector ranking.

Table 14. How the reported evidence should be used.

Evidence layer	Question answered	Allowed conclusion	Not a substitute for
Track-A L1	Surface robustness on fixed in-domain spam	Observed FNR under one deterministic transformation	Semantic LLM robustness
Reviewed L2/L3	Bounded local paraphrase behavior	Observed miss rate after validity controls	Unrestricted adaptive attack estimate
Track-B transfer	Sensitivity to new corpus distributions	Source-trained target performance	Deployment acceptance test
Target adaptation	Value of the stated local label budgets	Outcome on the fixed independent target test	A universal labeling rule
Latency microbenchmark	Relative modern-control CPU cost	Batch-one timing on stated VM	End-to-end service SLO

Table 14. How the reported evidence should be used.

Evidence layer	Question answered	Allowed conclusion	Not a substitute for
Track-A L1	Surface robustness on fixed in-domain spam	Observed FNR under one deterministic transformation	Semantic LLM robustness
Reviewed L2/L3	Bounded local paraphrase behavior	Observed miss rate after validity controls	Unrestricted adaptive attack estimate
Track-B transfer	Sensitivity to new corpus distributions	Source-trained target performance	Deployment acceptance test
Target adaptation	Value of the stated local label budgets	Outcome on the fixed independent target test	A universal labeling rule
Latency microbenchmark	Relative modern-control CPU cost	Batch-one timing on stated VM	End-to-end service SLO

7. Operational Implications, Replication, and Future Evaluation

7.1. Decision Process for a Local Deployment Candidate

A sparse detector should be evaluated as a candidate component in a mail-processing workflow, not as a stand-alone claim of security. The operational decision begins with the false-positive tolerance, the allowed miss rate, the expected language and source mix, and the local compute budget. A sparse source-trained model may be appropriate for a constrained on-premise deployment, but an organization should first collect a legally permitted local calibration set, remove cross-partition duplicates, and reserve a final target test set that is not opened during threshold selection.

Organizational studies of LLM-generated phishing and quishing show why a content-only offline score should not be read as an organization-level exposure estimate [48]. The operational implications here are therefore limited to the retained corpus and stated local workflow assumptions.

The measured results suggest a practical sequence. First, reproduce a clean target baseline with the frozen source model to make distribution shift visible. Second, decide whether target-domain labels are available and, if so, use an isolated adaptation pool rather than the final target test set. Third, choose a threshold by a stated operational objective such as maximum allowed FPR or FNR; this study does not claim that the default 0.5 probability threshold is universally optimal. Fourth, evaluate both clean and prespecified attack conditions, and record the selected operating point. Finally, monitor post-deployment drift and route ambiguous cases to a review or a higher-cost control when the risk policy permits.

7.2. What Must Be Reproduced Before a Comparison Is Credible

A future comparison should reproduce the data version and not only the model name. Minimum requirements are the archive identity, deduplication function, fixed ID list, label mapping, text-extraction policy, train/validation/test boundaries, random seed, vectorizer settings, classifier settings, calibration procedure, attack parameters, and score-to-label threshold. A separate manifest should identify every message removed for duplication or parsing failure so that an apparent performance gain can be traced to a population change.

For modern embedding controls, the model identifier alone is also insufficient. The tokenizer, maximum length, pooling policy, feature normalization, logistic-regression configuration, CPU thread count, warm-up count, batch size, and timing-clock boundaries affect both performance and latency. The present experiment calls these pipelines “embeddings plus logistic regression” precisely to prevent readers from interpreting the result as end-to-end fine-tuning. Such distinctions make the comparison replicable even when a future study substitutes a newer encoder.

Table 15. Pre-deployment reproduction checklist.

Stage	Required control	Failure prevented
Data intake	Hash archives and retain extraction report	Unknown or changed corpus
Partitioning	Split and group-deduplicate before fitting	Train-test overlap
Fitting	Fit vocabulary, calibration and augmentation on training side	Test information leakage
Attack evaluation	Freeze originals, parameters and validity decision	Changing denominator or meaning
Reporting	Retain predictions and paired statistics	Unverifiable aggregate claim
Performance	State CPU, threads, batch, warm-up and timing scope	Non-comparable latency

Table 15. Pre-deployment reproduction checklist.

Stage	Required control	Failure prevented
Data intake	Hash archives and retain extraction report	Unknown or changed corpus
Partitioning	Split and group-deduplicate before fitting	Train-test overlap
Fitting	Fit vocabulary, calibration and augmentation on training side	Test information leakage
Attack evaluation	Freeze originals, parameters and validity decision	Changing denominator or meaning
Reporting	Retain predictions and paired statistics	Unverifiable aggregate claim
Performance	State CPU, threads, batch, warm-up and timing scope	Non-comparable latency

7.3. Stronger Evidence and Realistic Workload Evaluation

Future evaluation should extend the retained protocol rather than substitute a new, less auditable benchmark. Candidate extensions include source-authorized corpus releases, end-to-end fine-tuned controls under a common compute budget, and a service-level workload that includes MIME/HTML parsing, queueing, concurrent workers, cache behavior, model loading, alerting, tail latency, and failure handling. Such a study should state the local cost of false positives and false negatives before selecting an operating threshold. Across extensions, a zero observed count must retain its interval, a curated target must retain its provenance label, a non-significant paired result must remain inconclusive, and values from different hardware must not be converted into a common latency ranking.

8. Threats to Validity and Limitations

8.1. Construct and Internal Validity

Spam labels, email text extraction, and the definition of a valid paraphrase are constructs that can be imperfect. Exact and group-near duplicate policies reduce a concrete leakage risk but cannot prove that every related campaign or template has been removed. The pipeline addresses a more direct internal-validity risk by freezing splits before augmentation and vectorizer fitting, and by using frozen paired prediction vectors for uncertainty analysis. Calibration and gate configuration are described as training-side operations; they must not be re-tuned from the evaluation files.

8.2. External Validity

Apache SpamAssassin, Enron, and the third-party curated TREC version are historical English-language collections with collection-specific label and formatting characteristics. They do not represent contemporary multilingual mail, attachment-heavy abuse, header manipulation, HTML rendering, or enterprise mail-routing policies. The TREC evaluation used a Zenodo-hosted curated CSV whose retained labels were not independently reconciled against an official raw-corpus mapping. Results are intentionally scoped to the retained data version and manifests.

Recent benchmark reassessments identify the historical age of widely used phishing-email data, incomplete realism, and conflation risks as material threats to external validity, including for SpamAssassin-derived evaluations [14]. This study does not dispute those concerns and does not present Track-A results as a general estimate of contemporary spam or phishing detection. Instead, SpamAssassin is used to demonstrate an auditable protocol with fixed manifests and paired prediction records; the Track-B cohort-policy comparison shows that a single benchmark cannot settle a broad effectiveness claim.

8.3. Attack and Deployment Validity

L1 is deterministic and therefore reproducible, but it is not a natural-language adversary. L2 and L3 use a local generator with retained decoding settings and a bounded query budget, but its human-readable release label, full prompt text, and complete candidate-selection policy were not retained. They do not support claims about proprietary models, retrieval-augmented attackers, tool use, unlimited adaptive search, or deliberate semantic restructuring. Similarly, the latency protocol is a single-CPU, batch-size-one microbenchmark with fixed inputs. It is not a service-level workload test and omits concurrent requests, message parsing from mail servers, network queues, and model-loading amortization.

8.4. Reproducibility and Ethics

The study retains audit outputs so readers can check aggregate numbers. Public corpora and third-party derived files require license and provenance checks before redistribution. Attack artifacts should use access controls because they support evaluation and could otherwise aid phishing. The protocol includes only the detail needed to reproduce measurement.

8.5. Retained-Scope Limitations

All reported results use one seed (42). The Track-A gated-versus-linear-SVM comparison is unresolved (p=0.5078), whereas the Track-B source-holdout comparison yields p=0.0215 before Holm adjustment and 0.0859 after it; stability across seeds was not tested, so no seed-level claim is made. No per-category easy_ham, hard_ham, spam, or spam_2 result is reported; aggregate F1 is not a per-category estimate. Gate-trigger prevalence on natural traffic was not logged, so no amortized gate cost is estimated.

No same-machine RMF-PC latency record was retained. The timing table therefore compares the two modern embedding controls only and supports no cross-model latency ranking. L2/L3 remain protocol demonstrations: they use small human-adjudicated candidate sets and do not estimate a generator-wide evasion rate. Finally, the Track-A MiniLM prediction vector was not retained, so no MiniLM Track-A metric or paired comparison is reconstructed. These limitations delimit the claims above rather than constitute unreported negative results.

9. Conclusion

This paper combines in-domain L1 evaluation, human-adjudicated local L2/L3 candidates, cross-corpus transfer, limited target-domain adaptation, and same-CPU embedding controls. Freezing cohorts, splits, attacks, review decisions, and paired prediction vectors keeps observed model differences tied to identifiable protocols. RMF-PC performed well under the named protocols, but the study does not cover unrestricted LLM-generated spam.

For the exact-deduplicated Track-A split, the measured L1 FNR for gated RMF-PC was 1.971%, and its paired difference from linear SVM was not statistically conclusive under the stated exact McNemar and paired-bootstrap analyses (p=0.5078; interval -1.613 to +0.538 percentage points). The same comparison in the separately constructed group-near-deduplicated Track-B source holdout gave a raw p=0.0215 and a negative interval, but a Holm-adjusted p=0.0859. This reversal does not establish superiority; it demonstrates that cohort policy can change an inferential conclusion and must be treated as an experimental variable. For human-adjudicated local attacks, one L2 miss was observed in 38 effective candidates and no L3 misses were observed in 55 effective candidates; the associated interval bounds require cautious interpretation. Across the separate Track-B cohort, source-trained results changed materially by target corpus, while the stated 5% and 10% target-label budgets improved the results on fixed independent target tests.

Future work should preserve corpus provenance, prevent split leakage before feature fitting or augmentation, report attack eligibility separately from generator yield, use paired uncertainty analyses for shared examples, and state hardware boundaries for latency. These records show whether a changed result came from the detector, the corpus, or the claim.

Table 16. Final claims and their evidence limits.

Claim	Evidence retained	Limit that remains
Track-A L1 behavior	Exact-deduplicated split and frozen paired predictions	One corpus, seed and deterministic attack
L2/L3 local attack outcome	Two-reviewer adjudicated effective candidate sets	Small samples and bounded local generator
Cross-corpus sensitivity	Separate group-near-dedup transfer outputs	Curated rather than official raw TREC target
Adaptation benefit	Fixed 5%/10% label budgets and independent target tests	No universal target-label recommendation
Modern-control cost	Same-CPU microbenchmark for two embedding controls	No same-machine RMF-PC latency claim

Table 16. Final claims and their evidence limits.

Claim	Evidence retained	Limit that remains
Track-A L1 behavior	Exact-deduplicated split and frozen paired predictions	One corpus, seed and deterministic attack
L2/L3 local attack outcome	Two-reviewer adjudicated effective candidate sets	Small samples and bounded local generator
Cross-corpus sensitivity	Separate group-near-dedup transfer outputs	Curated rather than official raw TREC target
Adaptation benefit	Fixed 5%/10% label budgets and independent target tests	No universal target-label recommendation
Modern-control cost	Same-CPU microbenchmark for two embedding controls	No same-machine RMF-PC latency claim