The Stale Gate: Pricing and Diagnosing Drift in Judge-Gated AI Systems

1. Introduction

LLM judges increasingly do more than evaluate outputs [17,19]: their scores can determine whether a system accepts a candidate, escalates to a stronger model, releases an artifact, or stores a trace for later reuse. These decisions are often governed by a fixed threshold calibrated under one judge, prompt, verifier, and workload. After any of those conditions change, the numerical threshold can remain unchanged even though the relationship between the score and correctness has shifted. A gate can therefore remain operational after the evidence that justified it has become stale. The resulting problem is temporal rather than purely statistical: when should a score stop controlling an action?

Several lines of work address parts of this problem. Reject-option classifiers and cascaded inference determine when uncertain cases should be routed to stronger systems [3,4]; calibration methods characterize the relationship between stated confidence and empirical outcomes [7]; studies of LLM judges document overconfidence, prompt sensitivity, and meta-evaluation failures [6,12,17]; and selective judge systems abstain or escalate under a calibrated snapshot [8,9]. Together, these approaches explain how score-based decisions can be constructed and assessed at a given operating point. A deployed gate introduces an additional question because its authority persists through time. Once the judge or workload changes, a system must determine what the resulting drift costs, how much audit evidence warrants a transition, and whether the supporting probability claim failed or the performance of the accepted cohort deteriorated. Those distinctions determine whether the appropriate response is recalibration, broader investigation, or temporary fail-safe routing.

Here, we treat the score gate, rather than the judge alone, as the scientific object: a rule that converts a claimed pass probability into an operational action. The score is therefore an empirical claim tied to a particular judge, prompt, verifier, workload, threshold, and audit policy. We attach two auditable contracts to this claim. The claim contract asks whether accepted outcomes remain consistent with the probabilities asserted by the judge. The performance contract asks whether the accepted cohort continues to meet a declared reference level. Their joint state identifies which evidence supporting the action has expired without assigning a unique physical cause, separating the need for score recalibration from degradation that requires investigation of the broader operating system.

We combine decision theory, anytime-valid monitoring, controlled interventions, and prospective API studies to test this view of retained score authority. We find that additive overconfidence has a predictable decision cost, that predeclared restart banks preserve a global false-alarm budget while recovering sensitivity to late drift, and that the two contracts produce distinct operational signatures. The external studies reveal a further constraint: the same careful-to-terse judge edit produces severe degradation on GSM8K, a much smaller effect on MBPP, and no claim-contract transition when an arithmetic workload becomes harder while the asserted probabilities remain compatible with audited outcomes. Together, these findings indicate that score authority is a property of the versioned relationship among a score, its outcomes, the workload, and the action it controls, rather than a permanent property of the numerical threshold itself.

This work makes three contributions:

• A decision-theoretic and sequential framework for stale gates. We derive an exact cost law for additive overconfidence and combine it with separate claim and performance monitors whose predeclared restart structure preserves anytime false-alarm control while recovering sensitivity to late drift (Section 3).
• Controlled diagnosis of expired score authority. We show that the joint contract signature distinguishes a stable gate, an overconfident judge, honest workload hardening, and a masked calibration failure while remaining explicit that the signature does not identify a unique causal mechanism (Section 4 and Section 5).
• Prospective evidence across generated and public tasks. Across 80 API-served deployments, we test the same control logic under generated arithmetic and code tasks and on MBPP and GSM8K, showing that the operational consequence of one judge edit is domain-specific (Section 5.3).

2. Related Work

Routing, calibration, and selective judging.

Reject-option classifiers and cascaded inference establish the basic decision problem of routing uncertain cases to more capable paths [3,4]. Predictor calibration provides a statistical basis for interpreting confidence scores [7], while studies of LLM judges show that those scores can be overconfident, prompt-sensitive, or unreliable under meta-evaluation [6,12,17]. Selective judge systems use calibrated uncertainty to abstain or escalate at a given snapshot [8,9]. These lines of work therefore explain how a score can govern action under stated operating conditions. We study the next lifecycle question: after that score has acquired operational authority, what evidence is required for the gate to retain that authority as the judge, verifier, workload, or cost structure changes?

Sequential evidence for change.

Sequential change detection provides tools for deciding when an observed stream has departed from a reference regime, including CUSUM and nonparametric e-detectors [13,15,16]. E-processes provide time-uniform evidence through Ville’s inequality and safe anytime-valid inference [11,14,18], and general e-detectors can aggregate processes started at multiple times. Our goal is not to introduce a new generic changepoint method or rank these approaches. Instead, we use anytime-valid evidence as the monitoring layer of a gate-specific control problem: decision costs define why drift matters, separate claim and performance contracts identify which operational promise has failed, predeclared restarts recover sensitivity to late change, and a fail-safe policy determines what the system is permitted to do next.

3. Pricing and Monitoring Score-Gated Decisions

To decide whether a score gate should retain authority, we connect two questions that are often treated separately. The first is consequential: how much does a change in the relationship between score and correctness alter the expected cost of routing? The second is evidential: does the information supporting the gate still hold, and which part of that support has failed? We first derive the calibrated decision boundary and the cost of additive overconfidence, then monitor the judge claim and accepted-cohort performance as separate contracts.

3.1. Pricing the Gate

Let a task have cheap-tier pass probability p, and let the judge report claimed pass probability q. The gate accepts the cheap path when $q\ge \tau$ and otherwise escalates to a strong path. Both routes incur $K_{\mathrm{cheap}}+K_{\mathrm{verify}}$. Escalation adds $K_{\mathrm{strong}}$, whereas an accepted failure adds loss L; within this decision model, the strong path is treated as successful. This setup makes the threshold a cost crossover rather than an arbitrary confidence cutoff.

Proposition 1 

(Calibrated Crossover). For a calibrated judge, $q=p$, expected cost is minimized by accepting when

$$p\ge {\tau }^{*}=1-{\displaystyle \frac{K_{\mathrm{strong}}}{L}}.$$

(1)

For $(K_{\mathrm{cheap}},K_{\mathrm{verify}},K_{\mathrm{strong}},L)=(1,0.5,8,20)$, the calibrated threshold is ${\tau }^{*}=0.60$. To examine stale authority while leaving the numerical gate unchanged, we introduce additive overconfidence $q_{\epsilon }=\min (p+\epsilon ,1)$. The reported claim shifts upward while the underlying pass probability does not, so tasks down to $p={\tau }^{*}-\epsilon$ become newly accepted. For score density f, the resulting incremental expected cost is

$$R(\epsilon )=L{\int }_{{\tau }^{*}-\epsilon }^{{\tau }^{*}}({\tau }^{*}-p)f(p)dp=\frac{1}{2}Lf({\tau }^{*}){\epsilon }^2+o({\epsilon }^2).$$

(2)

Equation (1b) isolates the operational price of drift by charging only the tasks that become newly admitted after the score shifts. Near the calibrated boundary, the local expansion is quadratic in the overconfidence magnitude, so small errors can initially have limited cost and then grow rapidly as a wider region below the original crossover is accepted. The important point is that the threshold itself need not move for the decision to change: drift changes which tasks receive cheap-path authority.

3.2. Monitoring Two Contracts with Restart Banks

Pricing establishes the consequence of stale authority, but it does not diagnose why the gate became stale. A judge can become overconfident while the accepted cohort still meets its service level; conversely, a workload can become harder while the judge remains calibrated. Because these cases require different responses, we monitor the probability claim and the performance of the routed cohort as separate contracts.

For each audited accepted task t, let $X_t\in \{0,1\}$ denote verified success and $q_t$ the judge claim. The first contract asks whether the judge is systematically promising more success than the audited outcomes support,

$$\mathbb{E}\left[X_t\mid {\mathcal{F}}_{t-1},q_t\right]\ge q_t.$$

(3)

The second contract asks whether the routed cohort continues to meet the declared reference level $m_0$,

$$\mathbb{E}\left[X_t\mid {\mathcal{F}}_{t-1}\right]\ge m_0.$$

(4)

We monitor either contract with the same segment-level construction. For predictable reference $r_t\in \{q_t,m_0\}$, a segment started at audit s uses

$$E_{s,t}=\exp \left({\displaystyle \sum _{i=s}^t}\lambda (r_i-X_i)-(t-s+1){\lambda }^2/8\right).$$

(5)

Hoeffding’s lemma [10] makes $E_{s,t}$ a nonnegative supermartingale under the corresponding conditional null, so Ville’s inequality provides time-uniform control of the probability that a segment ever crosses its boundary. A process started only once at deployment can, however, accumulate a long healthy prefix and respond slowly when drift begins late. To retain sensitivity to later changes without informally resetting the test, we predeclare starts $s=1,\dots ,H$. Within contract family j, the weights $w_{j,s}$ sum to at most one, and the family budgets satisfy ${\alpha }_{\mathrm{claim}}+{\alpha }_{\mathrm{perf}}=\alpha$.

Proposition 2 

(Joint Restart-Bank Validity). If Equations (2) and (3) hold, alarming whenever any segment reaches $1/({\alpha }_j{w}_{j,s})$ has probability at most α across both contract families and all predeclared starts. The result follows from Ville’s inequality and two union bounds; independence is not required.

Under a persistent mean gap g, expected log-evidence grows at first order as $\lambda g-{\lambda }^2/8$, giving

$$n_{\mathrm{delay}}\approx {\displaystyle \frac{\log \left(1/({\alpha }_j{w}_{j,s})\right)}{\lambda g-{\lambda }^2/8}};\lambda =4g\Rightarrow n_{\mathrm{delay}}\approx {\displaystyle \frac{\log \left(1/({\alpha }_j{w}_{j,s})\right)}{2g^2}}.$$

(6)

Equation (5) is an audit-budget approximation rather than a stopping-time bound, but it makes the evidence requirement interpretable in the same units used by deployment: audited accepted labels. Together, the pricing and monitoring components turn a static threshold into a versioned decision rule. The pricing law quantifies the cost of stale routing, the claim bank tests whether the score retains probabilistic authority, the performance bank tests whether the routed cohort retains its declared operating level, and the restart schedule determines how evidence from the current state can replace evidence accumulated under an earlier one.

4. Experiments

To determine whether stale score authority can be priced, diagnosed, and monitored under deployment-like conditions, we evaluated the framework in three layers. The first isolates the cost and detection problem under controlled drift, the second tests whether the two monitored contracts distinguish different failure modes, and the third asks whether the same logic remains informative when a hosted judge is used across different task domains.

Controlled calibration drift.

We first isolated a setting in which the source and magnitude of drift were known. We drew $p\sim Beta(2.2,1.8)$, used calibrated claims before drift, and introduced additive overconfidence afterward. Exact integration was paired with 300 Monte Carlo populations of 20,000 tasks, allowing the decision-cost law to be checked independently of sampling variation. Separate simulations then measured no-drift behavior, detection delay across five shift magnitudes, and sensitivity to drift beginning at epochs {0, 4, 8, 12}. The deployment or stream, rather than the audited label, was the statistical unit. This layer therefore tests whether the framework prices a known calibration error correctly and whether restart-based evidence remains usable when that error begins late.

Contract diagnostics.

Having established monitor behavior under one controlled drift mechanism, we next asked whether the two-contract architecture could distinguish qualitatively different losses of authority. We ran 1,000 deployments per intervention and divided $\alpha =0.05$ equally between the claim and performance families. Four controlled worlds represented a stable gate, an overconfident judge, an honestly reported harder workload, and a calibration failure masked by an easier accepted cohort. The interventions were constructed so that the probability claim and accepted-cohort performance could fail separately or together, providing a direct test of whether the alarm signature identifies which operational promise has been violated.

Prospective external studies.

We then moved from known interventions to API-served tasks. Generated arithmetic used exact numeric outcomes, whereas generated code was evaluated by executing six hidden tests in a separate python -I subprocess. Fifty deployments compared controls, a careful-to-terse judge-prompt edit, and an arithmetic workload shift. We subsequently applied the same prompt edit to MBPP code problems with executable assertions [2] and GSM8K human-written arithmetic problems [5], adding 30 completed deployments. Across the external studies, calibration-fit and holdout rows were separated, drift onset was randomized over epochs {4, 5, 6, 7}, and calendar starts were fixed independently of onset. Together, these layers move from mechanism to externally served behavior, testing the economics, diagnostic signature, and domain dependence of stale score authority without treating one provider or workload as a universal deployment model.

Use of generative AI.

Generative AI tools were used in preparing this manuscript and its supporting software; the full disclosure is given in Appendix H.3.

5. Results

5.1. Drift Has a Predictable Price, and Restarts Recover Late Evidence

We first asked whether a fixed numerical threshold could acquire a predictable cost when the score it relies on becomes overconfident. Equation (1b) matched paired Monte Carlo at every tested shift, with a largest absolute discrepancy of $2.65\times {10}^{-4}$ cost units (Figure 1b). At $\epsilon =0.15$, exact incremental cost was 0.3274 per task and Monte Carlo gave 0.3276 (bootstrap 95% CI [0.3270, 0.3282]). Over the same shift, coverage increased from 0.438 to 0.659, accepted failure increased from 0.241 to 0.320, and expected gated cost increased from 8.112 to 8.439. Thus, the threshold itself did not need to move for its operational meaning to change: calibration drift altered the population admitted by the gate and produced a measurable increase in decision cost.

Pricing the consequence of drift is only useful operationally if the system can also detect when the supporting evidence has become stale. We therefore tested whether the restart bank behaved conservatively under no drift while recovering sensitivity to changes that began late. The bank alarmed in 16 of 5,000 no-drift deployments, corresponding to 0.0032 with exact 95% CI [0.0018, 0.0052]. The empirical no-drift rate is an implementation check; the anytime guarantee follows analytically from Proposition 2. Across five shifts, mean detection delay was 1.01–1.04 times the first-order prediction in Equation (5). For the primary shift beginning at epoch 8, the restart bank alarmed after 156 audited labels on average (bootstrap 95% CI [152, 159]), compared with 585 among post-onset alarms for the start-once process. At epoch-12 onset, post-onset alarm probability was 0.998 for the bank and 0.038 for start-once. Predeclared restarts therefore preserve sensitivity to late change despite a long healthy prefix, while retaining the same declared error budget.

5.2. Two Contracts Distinguish What Failed

Detection alone does not determine what has gone stale. We therefore asked whether the two monitored contracts could separate different failures of score authority. At the population level, the overconfident-judge world produced both a claim gap of 0.142 and a performance gap of 0.078. Honest workload hardening reduced accepted performance by 0.059 while leaving the claim contract satisfied, whereas the masked-calibration world produced a 0.122 claim gap while accepted performance remained above its reference. These interventions created the distinct contract states needed to test whether the sequential alarms recover the difference between a broken probability claim and deterioration of the routed cohort.

The alarm signatures followed those population-level differences (Figure 2a). The stable world produced no alarm in 1,000 deployments. Overconfidence produced both alarms in 999/1,000 deployments. Honest workload hardening produced a performance-only signature in 966/1,000 deployments, with a post-onset claim-bank alarm rate of 0.006. The masked calibration failure produced a claim-only signature in 999/1,000 deployments and no post-onset performance alarm. The two-bank design therefore adds diagnostic structure to change detection: it identifies which monitored promise lost support, while deliberately leaving the physical cause of that failure to subsequent investigation.

5.3. One Judge Edit, Different Domain Consequences

The controlled worlds establish interpretable signatures, but they do not show whether the same judge change has the same consequence across domains. We therefore applied one careful-to-terse prompt edit to different task families (Figure 2b,c). On generated arithmetic, the edit raised accepted failure from 0.018 to 0.305, increased coverage from 0.698 to 0.943, and produced a post-onset claim gap of 0.302 (95% CI [0.294, 0.310]). Claim and performance processes alarmed in 10/10 deployments, and the calendar bank crossed in the randomized onset epoch every time.

An honestly reported harder arithmetic mix provided the complementary control. Coverage contracted from 0.690 to 0.456, but the post-onset claim gap was only 0.010 ([0.006, 0.015]), and no guard crossed in any of 10 deployments. The routing distribution therefore changed substantially without creating evidence that the probability claim itself had failed. This distinction is important because a monitor that treats every workload shift as miscalibration would withdraw authority for the wrong reason.

The same prompt edit produced a smaller effect on code. On generated functions, the mean claim gap was 0.057 ([0.041, 0.072]) and the calendar bank did not cross in 10 deployments. On MBPP, the gap was 0.068 ([0.050, 0.087]), accepted failure increased from 0.113 to 0.184, and no score, performance, or calendar-bank process crossed in 10 deployments. GSM8K occupied a different regime: the gap was 0.424 ([0.407, 0.441]), accepted failure increased from 0.086 to 0.477, and both contract families alarmed at the randomized onset in 5/5 deployments. The identical judge edit therefore cannot be assigned a domain-independent risk. Its operational consequence depends on the relation among the judge, workload, calibration evidence, and action controlled by the gate, and must be measured in the domain where that score will retain authority.

6. Discussion

Our combined theoretical, controlled, and prospective results support a common conclusion: once a judge score controls an action, a calibration result is not a permanent property of the score. It is evidence that a particular score-action relationship held under a defined operating state. The decision-cost analysis shows why this distinction matters. Additive overconfidence changed which tasks crossed an unchanged threshold, increasing accepted failure and expected cost without any visible movement in the gate itself. The monitoring studies then showed that the loss of authority can take more than one form: the claim contract tests whether accepted outcomes remain consistent with the probabilities asserted by the judge, whereas the performance contract tests whether the accepted cohort still meets its declared reference. Together, these results connect statistical drift to a lifecycle question that static calibration does not answer: when should a score stop controlling an action?

Relationship to selective prediction and cascades.

Reject-option classifiers, cascaded inference, and selective judge systems determine whether to accept, abstain, or escalate under a calibrated operating state [3,4,8,9]. Our results extend this decision from a snapshot to a lifecycle. A threshold remains justified only while current evidence continues to support the score–outcome relationship relevant to the action it controls. The cross-domain experiments make this dependence concrete: the same careful-to-terse edit produced severe degradation on GSM8K, a smaller effect on MBPP, and no claim-contract alarm under an honestly reported harder arithmetic workload. Thus, the relevant deployment object is not a judge or prompt in isolation, but the versioned relationship among the judge, workload, verifier, threshold, and downstream decision.

Interpreting the two-contract signature.

The controlled interventions show why two contracts are useful and also define the limit of their interpretation. A claim-only alarm indicates that asserted probabilities no longer agree with audited outcomes even when accepted-cohort performance remains above its reference. A performance-only alarm indicates that the routed cohort has degraded while the probability claim remains compatible with observed outcomes. When both alarms occur, both relationships have deteriorated. This separation matters because a single performance monitor would miss the masked calibration failure, whereas the claim contract alone would not express the service-level degradation produced by honest workload hardening. However, the alarm signature is not a causal fingerprint. Changes in the judge, prompt, verifier, workload, or routing logic can produce similar residual patterns. The signature therefore identifies which contract has lost support and narrows the next investigation; it does not by itself identify the physical cause.

Anytime evidence and late drift.

Time-uniform evidence is attractive for gates that may run indefinitely because it avoids fixing a stopping time in advance [11,14,18]. Yet a single process can accumulate a long healthy prefix, making a later shift harder to detect. The predeclared restart bank addresses this specific lifecycle problem by opening fresh evidence segments while preserving the global false-alarm budget through weighted starts. In the tested setting, this structure recovered late-onset sensitivity that the start-once process lost, while the measured delays remained close to the first-order calculation. We do not interpret this as a new general changepoint method. Rather, it shows how sequential evidence can be organized around the continuing authority of a score gate. Audit cadence and restart schedule are therefore part of the operational contract: they determine how quickly evidence from an earlier state can be replaced by evidence from the current one.

Extending gate governance to adaptive systems.

The present experiments isolate score-gated routing, whereas longer-horizon agents introduce feedback between decisions and later evidence. Escalation, memory admission, tool selection, repair, or release can change the distribution of subsequent tasks and the state being audited. A prospective extension should therefore version claim and performance contracts across these state changes, reserve audit evidence that is not itself consumed by adaptation, and compare contract-triggered recalibration with fixed-threshold operation. Natural software repositories, organic production drift, additional providers, and imperfect strong-tier outcomes would test whether the domain-specific effects observed here persist when the gate participates in a changing system. Such studies would determine whether versioned score authority improves decision reliability beyond the bounded routing settings examined in this work.

7. Operational Implications

Operationally, the monitor is most useful as a transition rule rather than another dashboard statistic. A score controls an action, audits test the contracts that justify that action, and the resulting signature determines what evidence or intervention is required next. No alarm means only that the accumulated audit evidence has not crossed a declared transition boundary; it does not certify indefinite validity. Under the policy studied here, a claim-only alarm suspends score-based authority pending recalibration or replacement. A performance-only alarm instead directs investigation toward changes in the workload, solver, verifier, or accepted cohort. When both contracts fail, the declared policy can move the system to its fail-safe route while a broader investigation is performed.

In the cascade studied here, fail-safe operation means escalating to the strong path. The same control primitive could in principle guard candidate acceptance, critic/refiner routing, promotion into reusable program libraries, memory or skill admission, solver selection, harness mutation, or release in continuous agents. These are proposed integration uses, not completed open-ended-agent deployments. Under the primary cost model, escalating every task costs 9.5 units, a 1.061-unit premium over the shifted gate. This makes the cost of temporary safety explicit and allows it to be compared with the cost of continuing to rely on a stale gate while its supporting evidence is repaired.

A deployable policy card should therefore version the event predicted by q, the action controlled by $\tau$, the reference $m_0$, audit cadence, restart schedule, signature meanings, and fail-safe action. Binding these elements to a specific judge, prompt, verifier, workload, and cost model turns an undocumented threshold into reviewable authority with an expiry condition. After any material change, the card makes explicit whether earlier evidence still applies or whether a new calibration and audit cycle is required.

8. Limitations

Empirical scope is the first limitation. The prospective evidence spans controlled simulations, generated tasks, and public MBPP and GSM8K benchmarks, but it uses one provider and one primary model snapshot. It does not test natural software repositories, organic production drift, or independently operated sites. The benchmark intervals also describe task ordering and assignment within shared finite pools rather than uncertainty over freshly sampled task populations. These constraints limit the transport of the observed domain contrast; they do not establish that the same prompt edit or alarm behavior will recur across providers, models, or production workloads.

The formal guarantee has a separate boundary. Holdout fitting and shrinkage sensitivity reduce leakage in the external studies, but finite plug-in references do not certify the conditional null assumed by the e-process theorem. The performance contract is cohort-level, the decision-cost model assumes that the strong tier succeeds, and detection delay is measured in audited accepted labels rather than wall-clock time. Thus, the analytic false-alarm statement applies under its declared assumptions, whereas the external experiments provide operating evidence rather than an unconditional deployment guarantee. Deployments that violate these assumptions require a revised model or new evidence rather than inheriting the theorem automatically.

Reproducibility and provenance impose a third limit. The supplement records partial historical row-level response linkage and one MBPP candidate whose output depends on Python set iteration order. These issues limit claims of exact re-execution and external transport, while the reported sensitivity analyses leave the principal endpoints and alarm decisions unchanged.

9. Conclusion and Future Work

We introduced the stale gate as a way to treat score-based authority as evidence that can expire. The decision-cost analysis shows how additive score drift changes the expected cost of decisions made under an unchanged threshold; the restart banks organize anytime evidence so that late drift can still be detected; and the two-contract design distinguishes deterioration of the judge claim from deterioration of the accepted cohort. Prospective API studies then showed why these distinctions matter in practice: the same judge edit was severe on arithmetic and modest on code, while an honestly reported workload shift changed routing without violating the claim contract.

The next step is to test this control logic in natural repositories, organic drift, and independently operated sites, where gates interact with longer trajectories and less controlled changes in system state. Those studies should measure not only alarm behavior but whether contract-triggered recalibration or fail-safe routing improves subsequent decisions relative to fixed-threshold operation. Together, the present results establish a practical basis for treating score authority as versioned evidence rather than permanent permission: it can be audited while it remains in force, withdrawn when its supporting contract fails, and restored only after new evidence shows that the gate is again fit for the action it controls.

Appendix A.1. Optimal Calibrated Gate

The calibrated threshold follows directly from the two actions available to the gate. For a task with cheap-tier pass probability p, accepting the cheap path has expected cost $K_{\mathrm{cheap}}+K_{\mathrm{verify}}+(1-p)L$. Escalating instead has cost $K_{\mathrm{cheap}}+K_{\mathrm{verify}}+K_{\mathrm{strong}}$ under the scoped assumption that the strong tier succeeds. Acceptance is therefore optimal exactly when the expected loss from cheap-path failure does not exceed the additional cost of escalation:

$$(1-p)L\le K_{\mathrm{strong}},p\ge 1-K_{\mathrm{strong}}/L={\tau }^{*}.$$

For $(K_{\mathrm{cheap}},K_{\mathrm{verify}},K_{\mathrm{strong}},L)=(1,0.5,8,20)$, the calibrated threshold is ${\tau }^{*}=0.60$.

Appendix A.2. Exact Price of Additive Overconfidence

To isolate the consequence of score drift while keeping the numerical gate fixed, let $q_{\epsilon }=\min (p+\epsilon ,1)$. Additive overconfidence newly admits tasks with p in $[{\tau }^{*}-\epsilon ,{\tau }^{*})$. At pass probability p, accepting instead of escalating incurs excess expected cost $(1-p)L-K_{\mathrm{strong}}=L({\tau }^{*}-p)$. Integrating this excess cost over that newly admitted region gives

$$R(\epsilon )=L{\int }_{{\tau }^{*}-\epsilon }^{{\tau }^{*}}({\tau }^{*}-p)f(p)dp.$$

Writing $u={\tau }^{*}-p$ and using continuity of f at ${\tau }^{*}$ yields $R(\epsilon )=\frac{1}{2}Lf({\tau }^{*}){\epsilon }^2+o({\epsilon }^2)$. Thus, $\frac{1}{2}Lf({\tau }^{*})$ is the local quadratic coefficient and the second derivative at zero is $Lf({\tau }^{*})$. This derivation explains the local quadratic behavior reported in the main text (Table A1) without assuming that larger shifts remain in the asymptotic regime.

Table A1. Exact Pricing and Paired Monte Carlo Reconstruction. Confidence intervals bootstrap 300 independent task populations.

$\mathit{\epsilon }$	Exact R	MC Mean	MC 95% CI	Local Quad.	Coverage	Accepted Fail.	Total Cost
0.000	0.0000	0.0000	[0.0000, 0.0000]	0.0000	0.438	0.241	8.112
0.025	0.0095	0.0095	[0.00945, 0.00954]	0.0095	0.476	0.255	8.121
0.050	0.0379	0.0379	[0.03774, 0.03801]	0.0380	0.514	0.269	8.150
0.100	0.1491	0.1491	[0.14873, 0.14944]	0.1522	0.588	0.295	8.261
0.150	0.3274	0.3276	[0.32701, 0.32823]	0.3424	0.659	0.320	8.439
0.200	0.5622	0.5621	[0.56120, 0.56304]	0.6087	0.727	0.343	8.674

Table A1. Exact Pricing and Paired Monte Carlo Reconstruction. Confidence intervals bootstrap 300 independent task populations.

$\mathit{\epsilon }$	Exact R	MC Mean	MC 95% CI	Local Quad.	Coverage	Accepted Fail.	Total Cost
0.000	0.0000	0.0000	[0.0000, 0.0000]	0.0000	0.438	0.241	8.112
0.025	0.0095	0.0095	[0.00945, 0.00954]	0.0095	0.476	0.255	8.121
0.050	0.0379	0.0379	[0.03774, 0.03801]	0.0380	0.514	0.269	8.150
0.100	0.1491	0.1491	[0.14873, 0.14944]	0.1522	0.588	0.295	8.261
0.150	0.3274	0.3276	[0.32701, 0.32823]	0.3424	0.659	0.320	8.439
0.200	0.5622	0.5621	[0.56120, 0.56304]	0.6087	0.727	0.343	8.674

Appendix B.1. Single-Segment e-Process

The monitoring problem is to accumulate evidence against a versioned contract without fixing in advance when the system must stop and inspect the stream. For audited outcome $X_t\in [0,1]$ and predictable reference $r_t$, suppose the conditional mean satisfies $\mathbb{E}[X_t\mid {\mathcal{F}}_{t-1},r_t]\ge r_t$. Hoeffding’s lemma gives

$$\mathbb{E}\left[\exp \left\{\lambda (r_t-X_t)-{\lambda }^2/8\right\}|{\mathcal{F}}_{t-1}\right]\le 1.$$

The product of these factors from a declared start s is a nonnegative supermartingale with initial value one. Ville’s inequality therefore implies $Pr\left({\sup }_{t\ge s}{E}_{s,t}\ge 1/{\alpha }_s\right)\le {\alpha }_s$. The claim bank sets $r_t=q_t$ and tests $\mathbb{E}[X_t\mid {\mathcal{F}}_{t-1},q_t]\ge q_t$; the performance bank sets $r_t=m_0$ and tests $\mathbb{E}[X_t\mid {\mathcal{F}}_{t-1}]\ge m_0$. The two processes use the same mathematical construction but answer different operational questions: whether the stated probabilities remain supported and whether the routed cohort retains its declared service level.

Appendix B.2. Two-Family Restart Bank

A process started once can become insensitive to a late change after accumulating a long healthy prefix. We therefore predeclare multiple starts while preserving a common false-alarm budget. Within each contract family, the weights assigned to the predeclared starts sum to at most one. A union bound over starts gives family-wise false-alarm probability at most ${\alpha }_j$, and a second union bound yields

$$Pr(\mathrm{any}\mathrm{alarm}\mathrm{from}\mathrm{either}\mathrm{family})\le {\alpha }_{\mathrm{claim}}+{\alpha }_{\mathrm{perf}}=\alpha .$$

No independence between starts or between contract families is required. In the contract-diagnostic study, ${\alpha }_{\mathrm{claim}}={\alpha }_{\mathrm{perf}}=0.025$, $H=24$, and uniform weights, so every segment boundary is $\log (24/0.025)=\log (960)=6.8669$. The restart construction therefore recovers sensitivity to late drift while keeping the declared joint error budget explicit.

Appendix B.3. Delay Approximation

The same construction also provides a first-order estimate of the number of audited accepted labels needed after a persistent shift. If the mean residual is $g=\mathbb{E}[r_t-X_t]>0$, the expected log-evidence increment is $\lambda g-{\lambda }^2/8$. Dividing the log boundary by this drift gives the delay calculator. The gap-matched value $\lambda =4g$ maximizes the expected increment and yields

$$n_{\mathrm{delay}}\approx \log (1/{\alpha }_s)/(2g^2).$$

Overshoot, mismatch between the tuning value and realized gap, censoring, and nonstationarity prevent this expression from being a universal stopping-time bound. We therefore use it as an audit-budget approximation and test its empirical accuracy in Appendix C.

Appendix E.1. Design and Evidence Role

After establishing the mechanism under controlled interventions, we tested the same logic with API-served generation. Before the first experimental call, the protocol, task generator, executor, and analysis program were fixed and recorded in an author-held provider-timestamped receipt. The registration record supports chronology but is not an independent preregistration or independent attestation of the request body.

The study used claude-haiku-4-5-20251001 [1] as both solver and judge at temperature zero. Arithmetic tasks were generated multi-step word problems with exact answers. Code tasks requested one-expression Python functions from 14 seeded templates with sentinel and edge-case variants. Ground truth for code was whether the produced function passed six hidden tests in a separate python -I subprocess with a five-second timeout. This design provides executed ground truth for the declared tests; it is not a hardened security sandbox and does not establish that six tests fully specify every function.

Per domain, week zero contained 300 tasks (Table A6). Rows 1–200 fit a Platt map, while rows 201–300 were held out for the accepted-cohort performance reference $m_0$ and selection-optimism shrinkage $\delta =\max (0,\overline{q}-\overline{X})$ among accepted holdout rows. A 1,000-resample bootstrap supplied the recorded upper 95% value for $\delta$. Deployment used the batch mean calibrated claim minus $\delta$ as the claim reference. These steps reduce in-sample and selection bias relative to the earlier pilot, but finite plug-in fitting does not itself establish the conditional null.

Table A6. Week-Zero Calibration and Holdout References for Generated Tasks.

Domain	All-Task Pass	Fit	Holdout	Accepted Holdout	${\mathit{m}}_0$	$\mathit{\delta }$ [Upper 95%]
Arithmetic	0.6733	200	100	69	1.0000	0.0000 [0.0000]
Code	0.7067	200	100	65	0.9231	0.0070 [0.0844]

Table A6. Week-Zero Calibration and Holdout References for Generated Tasks.

Domain	All-Task Pass	Fit	Holdout	Accepted Holdout	${\mathit{m}}_0$	$\mathit{\delta }$ [Upper 95%]
Arithmetic	0.6733	200	100	69	1.0000	0.0000 [0.0000]
Code	0.7067	200	100	65	0.9231	0.0070 [0.0844]

Fifty deployments then ran epochs 2–9 with 100 tasks per epoch and 10 task seeds per condition. Arithmetic included a control, a careful-to-terse judge-prompt change, and a base-to-hard task-mixture change. Code included a control and the same judge-prompt change. Drift onsets were randomized per seed over epochs {4, 5, 6, 7}, while the calendar claim bank started at epochs {2, 5, 8} independently of onset. The deployment seed, rather than the task or API call, was the statistical unit for the reported intervals.

Appendix E.2. Results

The generated-task results reproduced the qualitative distinction seen in the controlled worlds (Table A7). The arithmetic prompt-change bank crossed at the randomized onset in every seed, while all three guards remained silent in controls. The harder arithmetic mix reduced coverage by at least 0.15 in every seed but produced no claim alarm. On code, the prompt edit created a mean post-onset gap of 0.0566 (95% CI [0.0408, 0.0724]), yet the calendar bank remained silent in all 10 deployments; one start-once process crossed late. Upper-$\delta$ sensitivity did not change any calendar-bank verdict. These measurements show that the same edit can have materially different operational consequences across task domains.

Table A7. Operating States and Empirical Alarms for Generated Tasks. Intervals are seed-level normal intervals over 10 deployments. Controls report pooled values because they have no onset.

Domain	Condition	Coverage	Accepted Failure	Post Claim Gap [95% CI]	Claim Start	Claim Bank	Performance
Arithmetic	Control	0.705	0.017	–	0/10	0/10	0/10
Arithmetic	Prompt change	0.698 → 0.943	0.018 → 0.305	0.302 [0.294, 0.310]	10/10	10/10	10/10
Arithmetic	Harder mix	0.690 → 0.456	0.014 → 0.032	0.010 [0.006, 0.015]	0/10	0/10	0/10
Code	Control	0.689	0.087	–	0/10	0/10	0/10
Code	Prompt change	0.701 → 0.689	0.077 → 0.107	0.057 [0.041, 0.072]	1/10	0/10	0/10

Table A7. Operating States and Empirical Alarms for Generated Tasks. Intervals are seed-level normal intervals over 10 deployments. Controls report pooled values because they have no onset.

Domain	Condition	Coverage	Accepted Failure	Post Claim Gap [95% CI]	Claim Start	Claim Bank	Performance
Arithmetic	Control	0.705	0.017	–	0/10	0/10	0/10
Arithmetic	Prompt change	0.698 → 0.943	0.018 → 0.305	0.302 [0.294, 0.310]	10/10	10/10	10/10
Arithmetic	Harder mix	0.690 → 0.456	0.014 → 0.032	0.010 [0.006, 0.015]	0/10	0/10	0/10
Code	Control	0.689	0.087	–	0/10	0/10	0/10
Code	Prompt change	0.701 → 0.689	0.077 → 0.107	0.057 [0.041, 0.072]	1/10	0/10	0/10

Appendix E.3. Raw-Response Audit and Evidence Boundary

We audited the retained provider records to determine which parts of the external study can be reconstructed independently from the stored execution record. The gzipped response log contains 81,200 valid JSON records, 81,200 unique provider request IDs, 81,200 unique message IDs, and one returned model identifier. It contains 40,600 solver and 40,600 judge calls, and token totals exactly match the study summary. The per-task file contains 40,000 rows organized as 400 condition-seed-epoch groups of 100, and all 27,809 accepted rows are audited.

All 40,000 task rows are reconstructable from their seeded generators and linked to complete generated ground truth. Under the frozen parser, judge-response multisets reconcile in all 400 condition-seed-epoch groups. Historical response content permits unique ex-post row linkage for 19,732/40,000 judge calls (49.33%) and 5,940/24,000 arithmetic solver calls (24.75%); repeated terse or incorrect outputs remain identifiable only at group level. The public-benchmark study instead records task, call, and prompt identifiers before every call, providing complete ex-ante linkage for those runs.

The prospective generated-task design therefore provides 50 task-seeded deployments, holdout references, randomized onsets relative to a fixed calendar bank, arithmetic and executed-code outcomes, and raw provider responses. Its scope nevertheless remains one provider and one primary model snapshot. Control counts and intervention detections provide external operating evidence, whereas the theorem’s false-alarm authority still depends on the stated conditional-null assumptions.

Appendix F.1. Design and Completion Lineage

We next asked whether the contrast between arithmetic and code persisted on public benchmarks. The public-benchmark experiments applied the same careful-to-terse judge edit to MBPP and GSM8K. The MBPP arm and the later GSM8K completion arm were each fixed before execution and recorded with author-held provider-timestamped receipts; the complete registration records are retained with the evidence artifact (Appendix H.2).

The first run completed all 20 MBPP deployments and was interrupted during the GSM8K control condition. Its raw log contains partial GSM8K control activity but no completed GSM8K deployment and no prompt-change call. A separately specified completion experiment used a new seed namespace and was recorded before any GSM8K prompt-drift outcome had been observed. Both experiments exclude 24 MBPP bring-up tasks and 16 GSM8K bring-up tasks. Independent reconstruction confirms the exclusions, week-zero pools, fit/holdout split, deployment permutations, randomized onsets, prospective commitments, and regenerated endpoint files.

Appendix F.2. Results

MBPP week zero had solver pass 0.7767, accepted-cohort reference $m_0=0.8846$, shrinkage $\delta =0.0047$, and recorded upper sensitivity 0.0778. Controls produced no alarm. Under the prompt edit, the claim-outcome gap was 0.0684 and no score, performance, or calendar-bank process crossed in 10 deployments (Table A8); upper-$\delta$ sensitivity preserved the result. This overlaps the generated-code gap of 0.057 and extends the measurable-but-non-alarming code result to a public crowd-sourced benchmark.

Table A8. Public-Benchmark Arms. Intervals are Student-t intervals over task-order seeds drawn from shared finite benchmark pools.

Benchmark	Condition	Deployments	Accepted Failure	Post Claim Gap [95% CI]	Score	Bank	Performance
MBPP	Control	10	–	–	0/10	0/10	0/10
MBPP	Prompt change	10	0.1132 → 0.1838	0.0684 [0.0498, 0.0870]	0/10	0/10	0/10
GSM8K	Control	5	–	–	0/5	0/5	0/5
GSM8K	Prompt change	5	0.0861 → 0.4767	0.4242 [0.4073, 0.4412]	5/5	5/5	5/5

Table A8. Public-Benchmark Arms. Intervals are Student-t intervals over task-order seeds drawn from shared finite benchmark pools.

Benchmark	Condition	Deployments	Accepted Failure	Post Claim Gap [95% CI]	Score	Bank	Performance
MBPP	Control	10	–	–	0/10	0/10	0/10
MBPP	Prompt change	10	0.1132 → 0.1838	0.0684 [0.0498, 0.0870]	0/10	0/10	0/10
GSM8K	Control	5	–	–	0/5	0/5	0/5
GSM8K	Prompt change	5	0.0861 → 0.4767	0.4242 [0.4073, 0.4412]	5/5	5/5	5/5

GSM8K week zero had solver pass 0.5033, $m_0=0.9423$, $\delta =0.0211$, and upper sensitivity 0.0887. Controls again produced no alarm. Under the prompt edit, the gap was 0.4242 and the score, performance, and calendar-bank processes crossed in the randomized onset epoch in all five deployments, with the same result under upper-$\delta$ sensitivity. Thus, the public benchmarks preserve the central domain contrast: the same edit is modest on code and large enough to trigger both contracts on grade-school arithmetic.

Appendix F.3. Raw Linkage, Code Execution, and Uncertainty

The public-benchmark records provide stronger row-level provenance than the historical generated-task logs. The MBPP and GSM8K raw logs contain 21,548 and 16,525 records, respectively. Provider request IDs, message IDs, and call UIDs are unique; returned model identities are constant; and every prompt hash independently reconstructs. Every materialized task UID appears in its raw log. The GSM8K study has 75 solver parse failures for which the frozen runner correctly suppresses the judge call, explaining 16,525 rather than 16,600 records.

Independent re-execution reproduces 9,891/9,900 recorded MBPP task outcomes under a fixed hash seed. The nine differences all arise from one candidate, max(set(s), key=s.count), whose output is hash-order-sensitive when several characters tie because the frozen subprocess did not pin PYTHONHASHSEED. Reclassifying every affected occurrence as pass, every occurrence as fail, or excluding the task moves the mean gap only from 0.0664 to 0.0707 and changes no alarm or endpoint verdict.

The reported t intervals treat deployment task-order seeds as the units of analysis, but each condition samples heavily from one finite benchmark pool: MBPP uses 480 of 500 test tasks per deployment and GSM8K uses 800 of 1,003 post-exclusion deployment tasks. The intervals therefore quantify order and assignment variability under this design rather than transport uncertainty across independently sampled task populations, providers, or sites. Finite plug-in calibration also remains outside the analytic conditional-null guarantee.

Appendix G.1. Ramp and Lambda-Mixture Delay

A deterministic ramp study replaced a step change with a linear increase to gap $g=0.08$. If D denotes the step-delay calculator and R the ramp length, measured delay followed $D+R$ within 2.8% over the tested range (Table A9).

Table A9. Gradual Ramp Delay at $g=0.08$.

Ramp R	Measured Delay	Predicted $\mathit{D}+\mathit{R}$	Ratio
117	360.6	351.0	1.027
234	477.3	468.0	1.020
468	696.5	702.0	0.992
936	1149.1	1170.0	0.982

Table A9. Gradual Ramp Delay at $g=0.08$.

Ramp R	Measured Delay	Predicted $\mathit{D}+\mathit{R}$	Ratio
117	360.6	351.0	1.027
234	477.3	468.0	1.020
468	696.5	702.0	0.992
936	1149.1	1170.0	0.982

A five-$\lambda$ mixture avoids committing to a single target gap. Across gaps 0.05, 0.08, 0.12, and 0.20, its mean delay was 1.280, 1.276, 1.240, and 1.224 times the gap-matched process, respectively. These values quantify a tuning-robustness premium; they are not a free guarantee against misspecification.

Appendix G.2. Repeated Looks and CUSUM Comparisons

We compared the time-uniform construction with procedures that target different operating criteria. A per-batch binomial test spends $\alpha$ at every look and reached false-alarm rates 0.693 over 3,000 labels and 1.000 over 30,000 labels. The e-process had rates 0.040 and 0.0325 in those finite null simulations. A likelihood-ratio CUSUM threshold was Monte Carlo tuned for the 3,000-label horizon, where its false-alarm rate was 0.0325; carrying the same threshold to 30,000 labels produced 0.3975.

This comparison is not evidence that CUSUM is intrinsically defective. The procedures target different error criteria, and the CUSUM was not retuned for the longer horizon. At a changepoint after 1,500 healthy labels, the start-once e-process detected within horizon in 0.960 of recorded runs with mean delay 1,841 labels, whereas the reset-based CUSUM detected in 0.973 with mean delay 480. This negative control motivated the epoch-started bank and should not be interpreted as a general ranking of sequential methods.

Appendix G.3. Two-Sided Underconfidence Arm

A noisy-proxy stress test shifted confidence downward by 0.15 to examine the opposite direction of score error. Coverage fell from 0.435 to 0.219 and accepted failure from 0.248 to 0.157, while expected cost rose from 8.177 to 8.434. The upward-residual process alarmed in 0.92 of 50 shifted deployments and the downward process in 0.04; in 400 stationary deployments, the corresponding rates were 0.000 and 0.015.

These measurements are behavioral records rather than an inherited anytime guarantee because the score-derived reference in this arm was not re-certified under the conditional null after selection on a noisy proxy. The arm therefore illustrates the importance of re-establishing the contract assumptions when the reference itself is produced through an additional selection mechanism.

Appendix G.4. Additional API Sensitivity Studies

Additional API studies examined heterogeneity beyond the primary prospective sequence. A sample-consistency confidence measure under a temperature shift produced accepted failure 0.117 → 0.248 with recorded alarms. A stronger judge exposed to the same prompt edit showed a smaller increase, 0.000 → 0.053, without an alarm in the observation window. An engineered workload-mixture schedule kept accepted failure below 0.04 with all guards silent.

These studies reinforce dependence on the confidence construction, model strength, and task domain. They do not share the complete prospective and provenance contract of the primary external sequence and are therefore treated as sensitivity evidence rather than as additional confirmatory endpoints.

Appendix H.1. Reproducibility and Evidence Boundary

The records-first evidence artifact, available as described in Appendix H.2, is organized around the evidence required to inspect each reported result. For the controlled experiments, it contains the prospectively specified protocols and replicate-level records. For the generated-task API study, it includes protocols, receipts, raw responses, task rows, calibration rows, endpoints, and analysis programs. The MBPP and GSM8K materials additionally provide public-dataset provenance, per-task records, and independent verification. Registration metadata, checksums, and release lineage are retained with the evidence artifact (Appendix H.2) rather than repeated in the manuscript.

Offline verification checks manifests, task selection, prompt hashes, response linkage, endpoint reconstruction, and registered headline quantities without API access. Full controlled-study regeneration requires the pinned scientific Python environment. Re-executing an API study requires paid access to a mutable external service, so released raw responses serve as canonical execution records rather than evidence that future calls can be reproduced bit-for-bit. Reproduction of the analysis also does not independently authenticate provider request bodies.

Appendix H.2. Code and Data Availability

Analysis code, verification scripts, and checksums for the experiments reported in this work are available in a public repository at github.com/Daice-labs/stale-gate. The complete experimental records (task-level records, raw provider responses, endpoint files, and registration metadata) are retained by the authors and are available from the corresponding author on request.

Appendix H.3. Disclosure: Use of Generative AI

AI-assisted tools were used for language editing, implementation support, experiment orchestration, figure preparation, artifact review, and package assembly. Where an AI system formed part of an experimental procedure, its role is described in the corresponding methods. The named authors retain responsibility for the scientific questions, study protocols, source code, data provenance, analyses, citations, numerical claims, and final manuscript. AI systems are not authors.