Preprint
Article

This version is not peer-reviewed.

Trust as a Trajectory Property: An Epistemic Dynamics Framework for AI Evaluation

Submitted:

24 June 2026

Posted:

25 June 2026

You are already at the latest version

Abstract
Trustworthy deployment of AI systems under uncertainty requires more than accurate predictions: it requires structured mechanisms for confidence estimation, verification, escalation, and governance. We introduce a formal framework that models AI systems, human evaluators, and hybrid workflows as trajectories in an epistemic state space E defined by eight measurable variables: confidence, trust, uncertainty, verification effort, epistemic risk, cost, diversity, and transparency. State transitions are governed by a small algebra of transformations—judgment, verification, escalation—closed under composition on E, with initialization by a prediction generator and ensembling by an aggregation operator over evaluator collections. We separate admissibility axioms (cost accumulation, well-formedness) from stability conditions (calibration, risk monotonicity, diversity–risk coupling, verification efficacy, marginal cost), and define collapse modes as violations of the latter.Methods: The framework is operationalized through empirical estimators of all eight state variables from system logs, model outputs, and human feedback. We design four experiments testing five trajectory-level claims: MIMIC-IV ICU mortality prediction with clinician override as trust signal (n=3,000), multi-model LLM judge ensembles (simulated and real models via Ollama), recursive verification depth profiling, and pooled stability zone estimation.Results: All five claims are supported. Calibrated model outputs receive 16.7% more clinician trust (H1). In-basin trajectories yield 85.6% trust versus 55.4% out-of-basin (Cohen’s d=0.70, H4). Correlated error grows super-linearly as judge count decreases (b ≈ −1.9, H2). The stability functional peaks at finite verification depth n ∗ =4 (simulated) and n ∗ =2 (real LLMs), confirming diminishing returns to verification (H3). Real-model experiments confirm the self-critique accuracy degradation (88% to 56%) and cross-model recovery (90%).Conclusions: Trust is a trajectory-level property that cannot be reduced to point-wise accuracy. Stability zone membership provides a principled predictor of trust outcomes across clinical and LLM evaluation settings.
Keywords: 
;  ;  ;  ;  ;  ;  ;  ;  

1. Introduction

Modern AI systems are increasingly embedded in evaluation pipelines that combine model predictions, LLM-based judges, human oversight, and recursive verification layers. In clinical decision support, a model emits a recommendation, a clinician either trusts or overrides it, and a downstream auditor may review the override [1,2]. In LLM evaluation, a base model produces a candidate answer, a judge model scores it, and a meta-judge may adjudicate disagreements. Each of these settings is studied today under a different vocabulary: calibration, agreement, override rate, ensemble diversity, judge bias. What is missing is a unified representation under which trust emerges from the structured evolution of the pipeline rather than from any single model’s output. A prior series of works developed the conceptual foundations for this shift—from verification as the real cost of AI [3], through epistemic frontiers and technology discontinuities [4], collective multi-agent epistemics [5], and trust budgets for the optimal control of scientific discovery [6]—but stopped short of a unified dynamical-systems formalization.
Existing evaluation theory treats trust implicitly as a property of a model at a point in time. Calibration research assumes that a well-calibrated confidence score is sufficient for downstream decision-making [7,8,9]; clinical implementations of this idea have been developed for diagnostic transparency [1] and risk prediction [10]. LLM-as-a-judge work treats agreement among judges as a proxy for evaluation quality [11,12]. Ensemble methods characterize diversity as a property of a fixed model collection [13,14]. Governance frameworks for clinical AI have been proposed [15,16] and broadly reviewed [2,17], but without a dynamical model of trust trajectories. None of these accounts for the temporal, recursive, and governance-bound structure of real evaluation pipelines.
  • Gap.
There is no formal framework that (i) places models, judges, and human reviewers in a shared state space, (ii) defines operators that govern pipeline transitions, and (iii) characterizes pipeline failures as geometric regions rather than point-wise metrics. As a consequence, two systems with identical accuracy can exhibit vastly different trust behavior over time, and existing metrics cannot distinguish them.
Figure 1. Empirical trajectories overlaid on the stability landscape estimated from MIMIC-IV data ( n = 3 , 000 ) and simulated multi-model experiments. (A) Trajectories in the diversity–risk plane; the dashed contour marks the stability zone boundary ( Φ * = 0.10 ). In-basin points (green) cluster at low risk and high accuracy; out-of-basin points (red) span the full range. (B) Trust (accuracy proxy) histograms by stability zone membership. In-basin trajectories achieve 80.3 % accuracy versus 56.4 % for out-of-basin. This figure previews the paper’s central empirical finding: stability zone membership predicts trust outcomes. The framework that defines these zones is developed in the following sections. (A companion analysis using real clinician trust from MIMIC override logs yields d = 0.70 , Section 8.)
Figure 1. Empirical trajectories overlaid on the stability landscape estimated from MIMIC-IV data ( n = 3 , 000 ) and simulated multi-model experiments. (A) Trajectories in the diversity–risk plane; the dashed contour marks the stability zone boundary ( Φ * = 0.10 ). In-basin points (green) cluster at low risk and high accuracy; out-of-basin points (red) span the full range. (B) Trust (accuracy proxy) histograms by stability zone membership. In-basin trajectories achieve 80.3 % accuracy versus 56.4 % for out-of-basin. This figure previews the paper’s central empirical finding: stability zone membership predicts trust outcomes. The framework that defines these zones is developed in the following sections. (A companion analysis using real clinician trust from MIMIC override logs yields d = 0.70 , Section 8.)
Preprints 219974 g001
  • Contribution.
We address this gap with a dynamical-systems formulation of AI evaluation. Our contributions are:
  • We define an epistemic state space  E that uniformly represents AI systems, evaluators, and hybrid workflows (Section 3).
  • We introduce an operator algebra that separates a prediction generator P , three state transformations { J , V , G } , and an aggregation operator A over evaluator ensembles. The transformations are composable: any finite sequence yields a well-defined trajectory (Section 4).
  • We develop a stability theory that decouples admissibility (one axiom) from stability-defining conditions (five), defines collapse as the violation of any condition, and introduces a stability indicator Φ whose sub-level sets define stability zones (Section 5).
  • We operationalize all eight state variables as estimators computable from logs, model outputs, and human feedback (Section 6).
  • We instantiate four experiments on MIMIC-IV override logs, multi-model LLM judges, and recursive verification pipelines—using both simulation and real LLMs (Ollama: Llama 3.1, Phi 4, Qwen 2.5)—validating all five trajectory-level claims, including the central thesis that stability zone membership predicts trust ( d = 0.70 on n = 3 , 000 ICU stays) (Section 7 and Section 8).
The central thesis is summarized in one sentence:
Trustworthy AI systems are those whose epistemic trajectories remain within a bounded stability basin under recursive evaluation and bounded verification cost.
Figure 2. (Left) Epistemic state space ontology. A system occupies a state S = ( C , T , U , V , R , K , D , X ) . Operators P (prediction), J (judgment), V (verification), A (aggregation), G (escalation) govern transitions, forming trajectories T = ( S 0 , … , S t ) . Transformations { J , V , G } are closed under composition on E . Trust decisions occur at boundary τ , driven most directly by judgment and aggregation.(Right) The state space instantiated from MIMIC-IV data ( n = 3 , 000 ). Confidence C ^ vs clinician trust T ^ (jittered). Green: calibrated; red: miscalibrated. Blue line: mean trust per bin. Trust gap: + 0.167 (H1).
Figure 2. (Left) Epistemic state space ontology. A system occupies a state S = ( C , T , U , V , R , K , D , X ) . Operators P (prediction), J (judgment), V (verification), A (aggregation), G (escalation) govern transitions, forming trajectories T = ( S 0 , … , S t ) . Transformations { J , V , G } are closed under composition on E . Trust decisions occur at boundary τ , driven most directly by judgment and aggregation.(Right) The state space instantiated from MIMIC-IV data ( n = 3 , 000 ). Confidence C ^ vs clinician trust T ^ (jittered). Green: calibrated; red: miscalibrated. Blue line: mean trust per bin. Trust gap: + 0.167 (H1).
Preprints 219974 g002

3. Epistemic State Space

We define the ontology over which all subsequent dynamics are constructed. The eight-variable state space formalizes and extends the conceptual model introduced across the computational epistemics program [3] and builds on established frameworks for epistemic uncertainty decomposition [29].
 Definition 1
(Epistemic state space). The epistemic state space is the product space
E = [ 0 , 1 ] 6 × R ≥ 0 2 .
An epistemic state is an 8-tuple
S = ( C , T , U , V , R , K , D , X ) ∈ E ,
with the following coordinates:
  • C ∈ [ 0 , 1 ] — system confidence in its current claim;
  • T ∈ [ 0 , 1 ] — degree of trust extended by the downstream consumer;
  • U ∈ [ 0 , 1 ] — normalized epistemic uncertainty;
  • V ∈ R ≥ 0 — cumulative verification effort;
  • R ∈ [ 0 , 1 ] — normalized epistemic risk;
  • K ∈ R ≥ 0 — cumulative cost;
  • D ∈ [ 0 , 1 ] — epistemic diversity across evaluators;
  • X ∈ [ 0 , 1 ] — transparency of the current claim.
The partition into [ 0 , 1 ] and R ≥ 0 coordinates reflects that V and K are cumulative quantities that grow along a trajectory, while the remaining six are per-state intensities bounded per decision.
 Definition 2
(Accuracy functional). The accuracy functional a : E → [ 0 , 1 ] maps a state to the empirical correctness probability of the underlying claim, measured against ground truth whenever available and estimated by an independent gold-standard evaluator otherwise.

3.1. Admissibility

We require only one axiom of every trajectory. It states a basic well-formedness property (cost is monotone) and defines the universe of pipelines the framework can describe.
 Axiom 1
(Cost accumulation). Along any trajectory, cost is non-decreasing: K t + 1 ≥ K t for all t.
Axiom 1 is the only universal admissibility requirement. Trajectories that violate it (e.g., pipelines that fabricate negative cost) fall outside the framework’s scope.

3.2. Stability Conditions

The following five conditions are not assumed; they define stability. A trajectory may satisfy any subset of them, and the subset it violates identifies its collapse mode (Definition 11). Each condition is stated in the discrete form appropriate to the operator algebra of Section 4.
 Condition 1
(Calibration). For tolerance δ ≥ 0 , | C t − a ( S t ) | ≤ δ for all t.
 Condition 2
(Risk–uncertainty coupling). For any two states S , S ′ ∈ E differing only in the U coordinate, U > U ′ ⇒ R ≥ R ′ . This is a cross-sectional property of the state space (higher-uncertainty states have weakly higher risk), not a trajectory implication; it is compatible with operators such as V that reduce R at fixed U.
 Condition 3
(Diversity–risk coupling). There exist β , R 0 > 0 such that R t ≥ R 0 ( 1 − D t ) for all t. The bound is trivial at D t = 1 and forces R t ≥ R 0 when diversity is fully depleted.
 Condition 4
(Verification efficacy). Whenever V is applied between t and t + 1 , R t + 1 ≤ R t .
 Condition 5
(Marginal verification cost). There exists c min > 0 such that K t + 1 − K t ≥ c min · ( V t + 1 − V t ) whenever V or G is applied.
Conditions 1–5 define the five axes along which a trajectory can fail. Crucially, they are independent: a trajectory may be calibrated (Condition 1) while failing diversity (Condition 3), as in a single-model judge with accurate confidence but no evaluator disagreement. Section 5 formalizes how the conjunction of these conditions defines the stable basin and how each individual violation defines a named collapse mode.

4. Epistemic Dynamics

State transitions are governed by an operator algebra with three distinguishable roles: a generator that initializes a state from an external input, transformations that evolve a single state into its successor, and an aggregator that combines several evaluator states into one. The transformation set { J , V , G } is composable: any finite sequence of transformations applied to an initial state yields a well-defined trajectory (Proposition 1).

4.1. Operators

Let X denote the input space (e.g., patient features, prompts).

Generator.

 Definition 3
(Prediction generator). P : X → E maps an input x ∈ X to an initial state S 0 = P ( x ) with V 0 = K 0 = 0 and ( C 0 , T 0 , U 0 , R 0 , D 0 , X 0 ) determined by the model’s output distribution.
P is not in the operator algebra proper; it constructs S 0 and is then composed away.

Transformations ( E → E ).

 Definition 4
(Judgment operator). J : E → E produces an evaluation of the current claim. It writes ( C , X , U ) ← ( J C , J X , J U ) and preserves ( V , K , R , D ) . The preservation of R encodes that judging alone neither verifies nor mitigates risk.
 Definition 5
(Verification operator). V : E → E performs an external check. It increments ( V , K ) ← ( V + Δ V , K + Δ K ) with Δ V , Δ K > 0 , decrements R ← R − Δ R for some Δ R ≥ 0 , and preserves ( C , T , U , D , X ) .
 Definition 6
(Escalation operator). G : E → E triggers a higher-order review. It increments ( V , K ) ← ( V + Δ V esc , K + Δ K esc ) with Δ K esc ≥ Δ K for any single V application, may update ( C , R ) , and preserves ( T , U , D , X ) . The escalation operator formalizes the selective-prediction (reject-option) mechanism studied in Geifman and El-Yaniv [27] and extended to human-AI collaboration by Mozannar and Sontag [28].

Aggregator (over evaluator ensembles).

 Definition 7
(Ensemble). An ensemble of size n at time t is a collection Σ t = ( S t ( 1 ) , … , S t ( n ) ) ∈ E n of evaluator states produced by n parallel transformations applied to a common predecessor state.
 Definition 8
(Aggregation operator). A : E n → E maps an ensemble Σ t to a single consensus state S t + 1 = A ( Σ t ) . The diversity coordinate of S t + 1 is
D t + 1 = 1 − 2 n ( n − 1 ) ∑ i < j sim S t ( i ) , S t ( j ) ,
where sim : E × E → [ 0 , 1 ] is a domain-appropriate state similarity. The remaining coordinates of S t + 1 are obtained by an aggregation rule (e.g., weighted mean of ( C , R , U , X ) ; sum of ( V , K ) ).
The ensemble structure of Definition 7 is the natural locus for collapse mode (C) (Section 5): the inter-evaluator variance Var i ( S t ( i ) ) is well-defined on Σ t .

4.2. Trajectories

 Definition 9
(Epistemic trajectory). An epistemic trajectory of horizon T is a sequence T = ( S 0 , S 1 , … , S T ) with S 0 = P ( x 0 ) and
S t + 1 = O t ( S t ; x t , ξ t ) , O t ∈ { J , V , G } ,
where ξ t captures sampling and human-behavior stochasticity. Ensemble sub-trajectories are admitted at any t by replacing the single transition S t ↦ S t + 1 with a two-step transition
S t ↦ Σ t = ( J ( 1 ) ( S t ) , … , J ( n ) ( S t ) ) ↦ S t + 1 = A ( Σ t ) ,
i.e., parallel judgment by n evaluators followed by aggregation. The ensemble Σ t is observable at time t and supports measurement of inter-evaluator variance.
The canonical evaluation pipeline is now expressible as a composition of E → E transformations, optionally punctuated by an ensemble step:
T = P ( x 0 ) , J ∘ P ( x 0 ) , V ∘ J ∘ P ( x 0 ) , G ∘ V ∘ J ∘ P ( x 0 ) ,
or, with an ensemble at the second step,
T = P ( x 0 ) , A ∘ Σ [ J ( 1 ) , … , J ( n ) ] ∘ P ( x 0 ) , V ∘ … .
 Proposition 1
(Composability of the transformation set). The transformation set { J , V , G } is composable: any finite composition of transformations applied to an initial state S 0 ∈ E yields a well-defined trajectory in E .
Proof. 
Each transformation maps E → E by Definitions 4–6. The composition of measurable maps E → E is a measurable map E → E . Application to S 0 ∈ E therefore yields a well-defined state at every finite horizon. The aggregator A has signature E n → E and is admitted only at ensemble sub-steps where an E n -valued Σ t has been constructed by parallel transformations; it is not part of the E → E closure claim.    □
Figure 3. Empirical epistemic trajectory from the recursive verification experiment (Experiment 3, simulated graduated pipeline, 500 items). (A) Theoretical collapse regimes in the verification–confidence plane: well-calibrated region (green), overconfidence (red), monoculture (magenta), and over-verification (orange). (B) Empirical trajectory: confidence C ^ improves through depth 3, then saturates. (C) Risk–cost tradeoff: error rate R ^ decreases through depth 3 while cost K ^ grows linearly; optimal depth n * = 4 occurs at the benefit–cost crossover. Compare panel (A)’s theoretical regimes with the empirically observed path in (B)–(C): the trajectory traverses multiple regimes before settling in the well-calibrated region at moderate verification depth.
Figure 3. Empirical epistemic trajectory from the recursive verification experiment (Experiment 3, simulated graduated pipeline, 500 items). (A) Theoretical collapse regimes in the verification–confidence plane: well-calibrated region (green), overconfidence (red), monoculture (magenta), and over-verification (orange). (B) Empirical trajectory: confidence C ^ improves through depth 3, then saturates. (C) Risk–cost tradeoff: error rate R ^ decreases through depth 3 while cost K ^ grows linearly; optimal depth n * = 4 occurs at the benefit–cost crossover. Compare panel (A)’s theoretical regimes with the empirically observed path in (B)–(C): the trajectory traverses multiple regimes before settling in the well-calibrated region at moderate verification depth.
Preprints 219974 g003

5. Stability Theory

This section identifies the conditions under which a trajectory is stable and describes the collapse regimes that emerge when those conditions fail.

5.1. Stable Trajectories

 Definition 10
( ( ϵ , δ , D min ) -stability). Given constants ϵ , δ , D min > 0 , a trajectory T is ( ϵ , δ , D min ) -stable if at every t ∈ { 0 , … , T − 1 } all of the following hold jointly:
(i)
| R t + 1 − R t | ≤ ϵ         (bounded risk drift),
(ii)
Condition 1 holds with tolerance δ,
(iii)
D t ≥ D min       (non-degenerate diversity),
(iv)
Conditions 2 and 4 hold,
(v)
Condition 5 holds with c min .
Definition 10 is non-vacuous: a generic admissible trajectory (satisfying only Axiom 1) need not satisfy any of (i)–(v), since the transformation operators are not constrained a priori to preserve them. Stability is therefore a property of operator behavior, not of the axiom system.

5.2. Collapse Regimes

Collapse modes (A) and (B) are direct falsifications of Conditions 1 and 4 respectively. Collapse modes (C) and (D) are threshold-crossing events: they occur when a coordinate exceeds a critical value even though no condition is formally violated. All four are detectable by the empirical tests of Section 6.
 Definition 11
(Epistemic collapse). A trajectory T collapsesat time τ if at least one of the following holds:
(A) 
Overconfidence collapse (Condition 1 violation). C τ − a ( S τ ) > δ . The system is more confident than its accuracy warrants. The symmetric case a ( S τ ) − C τ > δ (underconfidence) also violates Condition 1 but is not a named collapse mode because it produces over-conservative behavior rather than unsafe trust; we note the asymmetry and restrict attention to the overconfidence direction.
(B) 
Verification breakdown (Condition 4 violation). A V step occurs at τ with R τ > R τ − 1 : verification was applied but risk increased.
(C) 
Monoculture collapse (diversity threshold). D τ < D min and, for the ensemble Σ τ admitted by Definition 7, Var i ( S τ ( i ) ) → 0 while R τ ≥ R 0 . Diversity is depleted and evaluators agree on errors. Note: Condition 3 remains formally satisfied (it forces R ≥ R 0 at low D); the collapse is the consequence of depleted diversity, not the violation of the bound.
(D) 
Over-verification divergence (cost–risk threshold). K τ − K τ − 1 > κ while | R τ − R τ − 1 | < η for fixed κ , η > 0 : cost grows without proportional risk reduction. Condition 5 remains satisfied; the collapse is the crossing of a cost-efficiency threshold.
Modes (A) and (B) co-occur when a miscalibrated verifier both overtrusts and amplifies risk. Modes (C) and (D) represent opposite failure extremes: insufficient diversity drives (C), while excessive verification drives (D).

5.3. Stability Indicator

States can be compared by how close they are to satisfying Conditions 1 and 3. We define a heuristic indicator that aggregates the per-condition penalties; it is one of several possible scalar summaries and should be interpreted as a diagnostic tool rather than a universal measure.
 Definition 12
(Stability indicator). The stability indicator Φ : E → R ≥ 0 is
Φ ( S ) = w 1 max 0 , | C − a ( S ) | − δ + w 2 max 0 , R 0 ( 1 − D ) − R + w 3 max 0 , D min − D ,
with weights w 1 , w 2 , w 3 > 0 .
Φ ( S ) = 0 iff S satisfies Conditions 1 and 3 with margins δ , D min . Sub-level sets { S : Φ ( S ) ≤ Φ * } define stability zones—nested approximations of the region where conditions hold. The zone boundary Φ * is a free parameter; we report sensitivity sweeps across Φ * values (Section 8) rather than asserting a single optimal threshold.

5.4. Stability Functional

To compare trajectories quantitatively, we define a scalar aggregate of four diagnostic quantities. This functional is a design choice, not a unique or optimal summary; alternative formulations (multiplicative, entropy-based, Pareto-frontier) could be equally valid. The cost term K ¯ is normalized by a per-experiment reference K ref to make it commensurable with the other three components.
 Definition 13
(Stability functional). For a trajectory T of horizon T, define
S ( T ) = α · Cal ( T ) − β · Δ R ( T ) − γ · Col ( T ) − δ w · K ¯ norm ( T ) ,
where α , β , γ , δ w > 0 with α + β + γ + δ w = 1 , and
Cal ( T ) = 1 T ∑ t = 0 T − 1 1 − | C t − a ( S t ) | ( mean calibration ) , Δ R ( T ) = max t | R t + 1 − R t | ( worst - case risk drift ) , Col ( T ) = 1 T ∑ t = 0 T − 1 1 D t < D min ( collapse exposure ) , K ¯ norm ( T ) = 1 K ref · 1 T ∑ t = 0 T − 1 K t ( mean cos t , normalized ) .
The reference cost K ref is a per-experiment constant (e.g., the median mean-cost across the trajectory pool, or a domain-specific per-decision budget) that maps K ¯ onto a scale comparable to Cal , Δ R , Col ∈ [ 0 , 1 ] . Its value is pre-registered per experiment to avoid post-hoc tuning.
 Proposition 2
(Optimal verification depth). Suppose the trajectory is parameterized by a verification depth n ∈ N , with R ( n ) strictly decreasing and convex in n and K ( n ) strictly increasing and convex. Then S ( n ) attains its maximum at a finite
n * = arg max n ∈ N S ( n ) < ∞ .
Proof. 
Under Axiom 1 and Condition 5, each V application adds at least c min · Δ V to K, so K ¯ ( n ) ≥ c min · n · Δ V → ∞ as n → ∞ . The term Col ( n ) is bounded in [ 0 , 1 ] , and Δ R ( n ) ∈ [ 0 , 1 ] since R ∈ [ 0 , 1 ] . The only unbounded component of S ( n ) is − δ w · K ¯ ( n ) , which tends to − ∞ . Therefore S ( n ) → − ∞ as n → ∞ . On the compact restriction n ∈ [ 1 , n max ] , S is continuous in n (each component is a finite sum or max of continuous functions) and attains a maximum by the extreme value theorem; the divergence at infinity forces the maximizer to be finite.    □
The earlier claim that S ( n ) is concave is withdrawn: it does not follow from the convexity of R ( n ) and K ( n ) alone. The finite-maximizer claim survives without it.
Figure 4. Stability landscape estimated from MIMIC-IV data ( n = 3 , 000 ). (A) Stability indicator Φ ( C , R ) heatmap over the confidence–risk plane, with the basin boundary Φ * = 0.10 marked (red contour). Green points: trajectories classified as in-basin ( Φ ≤ 0.10 ). Red points: out-of-basin. In-basin states cluster at moderate confidence ( C ≈ 0.5 – 0.8 ) with moderate risk ( R ≈ 0.1 ); the basin excludes very low confidence (uncertain predictions) and very high confidence (overconfident predictions). (B) Clinician trust distribution by basin membership. In-basin trajectories receive 85.6 % trust (mean) versus 55.4 % for out-of-basin trajectories (Cohen’s d = 0.70 , H4). The density shift is substantial: the in-basin distribution is concentrated near 1.0 , while the out-of-basin distribution is nearly uniform.
Figure 4. Stability landscape estimated from MIMIC-IV data ( n = 3 , 000 ). (A) Stability indicator Φ ( C , R ) heatmap over the confidence–risk plane, with the basin boundary Φ * = 0.10 marked (red contour). Green points: trajectories classified as in-basin ( Φ ≤ 0.10 ). Red points: out-of-basin. In-basin states cluster at moderate confidence ( C ≈ 0.5 – 0.8 ) with moderate risk ( R ≈ 0.1 ); the basin excludes very low confidence (uncertain predictions) and very high confidence (overconfident predictions). (B) Clinician trust distribution by basin membership. In-basin trajectories receive 85.6 % trust (mean) versus 55.4 % for out-of-basin trajectories (Cohen’s d = 0.70 , H4). The density shift is substantial: the in-basin distribution is concentrated near 1.0 , while the out-of-basin distribution is nearly uniform.
Preprints 219974 g004

6. Measurement Model

The state variables in Definition 1 are latent. We now define empirical estimators C ^ , T ^ , U ^ , V ^ , R ^ , K ^ , D ^ , X ^ computed from observable signals: model logits, judge outputs, override labels, and timing data.
 Definition 14
(Empirical estimators). Given a decision log L t = ( x t , y t , r t , τ t ) at time t, with model output y t , judge responses r t = ( r t ( 1 ) , … , r t ( n ) ) , and timing τ t , define:
C ^ t = 1 − H p θ ( · ∣ x t ) log K ( normalized entropy , K classes ) or verbalized confidence score , T ^ t = 1 − 1 override at t , U ^ t = Var r t ( 1 ) , … , r t ( n ) , V ^ t = number of verification calls up to t , R ^ t = P ^ ( error ∣ x t ) · c err ( x t ) , K ^ t = α lat τ lat , t + α cmp c cmp , t + α hum τ hum , t , D ^ t = 1 − 2 n ( n − 1 ) ∑ i < j sim r t ( i ) , r t ( j ) , X ^ t = # grounded tokens in y t # tokens in y t .
The estimator c err is a domain-specific error cost (e.g., clinical harm weight); similarity sim is computed over judge-output embeddings.
 Definition 15
(Empirical trajectory). The empirical trajectory induced by a log L = ( L 0 , … , L T ) is
T ^ ( L ) = S ^ 0 , S ^ 1 , … , S ^ T , S ^ t = C ^ t , T ^ t , U ^ t , V ^ t , R ^ t , K ^ t , D ^ t , X ^ t .
The estimators of Definition 14 are intentionally permissive: multiple operationalizations of each variable are admissible. The framework’s claims (Section 8) hold for any estimator family satisfying the axioms of Section 3.

6.1. Collapse Detectors

The four collapse modes of Definition 11 admit direct empirical tests.
Overconfidence : C ^ t − a t > δ .
Monoculture : D ^ t < D min and Var ^ ( S t ( i ) ) → 0 .
Verification breakdown : V applied at t and R ^ t > R ^ t − 1 .
Over - verification : K ^ t − K ^ t − 1 > κ while | R ^ t − R ^ t − 1 | < η .

7. Experimental Design

We test the framework via four experiments. Each is paired with a falsifiable hypothesis phrased in the language of the preceding sections. Experiments 1–3 probe individual operators and conditions; Experiment 4 tests the central thesis (basin membership predicts trust).

7.1. Experiment 1: Trajectory-Level Calibration and Trust (Clinical AI)

  • Data.
MIMIC-IV [22], restricted to clinical decisions with documented model recommendations and clinician override labels. The calibration–transparency framework of Yu et al. [1] provides feature-engineering precedents for this cohort.
  • Hypothesis H1 (trajectory-level).
Trajectories that satisfy Condition 1 at every step yield higher mean trust than trajectories with the same terminal accuracy but intermittent calibration:
E T ¯ | Condition holds ∀ t , a ( S T ) = a 0 > E T ¯ | intermittent calibration , a ( S T ) = a 0 ,
where T ¯ = T − 1 ∑ t T ^ t . This is stronger than a point-in-time conditional because it isolates trajectory structure from terminal accuracy.
  • Analysis.
(i) Calibration curve C ^ t vs. a t per decision. (ii) Trust trajectory T ^ t vs. override rate. (iii) Matched comparison of T ¯ across trajectory classes that share terminal accuracy. Note that the current experiment evaluates the cross-sectional implication of Claim 2 (per-stay calibration quality predicts trust); a full temporal trajectory test requires multi-window feature extraction within each ICU stay and is left as future work.

7.2. Experiment 2: LLM-as-a-Judge Monoculture and Condition 3 Validation

  • Data.
Simulated multi-model judge pool with pre-registered accuracy profiles and correlated bias categories (approximating a heterogeneous model zoo: GPT, Claude, Gemini, Qwen).
  • Hypothesis H2a (axiom validation).
The data are consistent with Condition 3: R ^ decreases with D ^ after controlling for accuracy. (This validates a condition of the framework, not a theorem.)
  • Hypothesis H2b (non-axiomatic prediction).
Correlated error—defined as the fraction of errors shared across all evaluators in Σ t —grows super-linearly as n decreases, even when mean confidence and accuracy are held constant. This is not entailed by Condition 3 alone and is the framework’s non-trivial prediction about monoculture.
  • Analysis.
Vary (i) n ∈ { 1 , 2 , 4 , 8 } , (ii) prompt similarity, (iii) temperature. Report D ^ vs. R ^ , agreement vs. correctness, correlated-error growth curves, and a judge-similarity heatmap.

7.3. Experiment 3: Recursive Verification Stability

Data.

Simulated graduated verification pipeline (self-critique → cross-model → RAG → committee → escalation) over a synthetic benchmark ( n = 500 items, Beta(2,2) difficulty distribution).

Hypothesis H3.

The stability functional is unimodal in verification depth: there exists a finite n * = arg max n S ( n ) < ∞ , consistent with Proposition 2.

Analysis.

Plot R ^ ( n ) , K ^ ( n ) , S ^ ( n ) as functions of n. Identify the empirical n * and test unimodality via the dip test.

7.4. Experiment 4: Basin Estimation and Trust Prediction

Data.

Pooled trajectories from Experiments 1–3.

Hypothesis H4 (central thesis).

Basin membership predicts trust outcomes: trajectories classified as in- B * yield higher mean T ^ than trajectories classified as out-of- B * , controlling for terminal accuracy:
E T ¯ | B * ^ - membership , a ( S T ) > E T ¯ | outside B * ^ , a ( S T ) .

Analysis.

(i) Estimate ∂ B * ^ by fitting the stability indicator Φ ^ (Definition 12) over observed trajectories and thresholding at Φ * . (ii) Classify each trajectory as in-basin or out-of-basin. (iii) Compare T ¯ distributions across classes with accuracy covariates. (iv) Report the area under the precision-recall curve for basin membership as a predictor of trust outcomes. This is the framework’s central falsifiable test (see Figure 1, introduced in Section 1).

8. Results

We organize results around five falsifiable claims. The first four probe individual components of the framework; the fifth tests the central thesis. Experiment 1 and Experiment 4 use real MIMIC-IV ICU data ( n = 3 , 000 test stays from a 10 , 000 -stay cohort with temporal split of a logistic regression mortality predictor). Experiments 2 and 3 combine pre-registered simulations with real-model validation using local LLMs (Ollama: Llama 3.1, Phi 4, Qwen 2.5, GPT-oss on n = 30 –50 items per configuration). All numerical results pass pre-registered statistical thresholds; full output is available in supplementary tables.
 Claim 1
(State space validity). The empirical trajectory T ^ ( L ) is well-defined and stable under bootstrap resampling of the log L .
  • Test.
Bootstrap L with B = 1000 resamples; report the bootstrap distribution of T ^ and its 95% confidence band. Result (Experiment 3, simulated): The trajectory estimator is stable under bootstrap resampling: the 95% CI band width for S ^ ( n ) is ≤ 0.01 at the peak n * = 4 (Figure 7).
 Claim 2
(Trajectory dependence of trust). Trust cannot be predicted from an instantaneous state alone; there exists a functional f such that T ¯ = f ( T ) and f is not constant on trajectories sharing the same terminal state.
  • Test.
Identify pairs of systems with identical terminal accuracy but divergent trajectories; compare T ^ distributions. Result (Experiment 1, MIMIC-IV):H1 supported. Calibrated confidence bins receive a trust premium of + 0.167 over miscalibrated bins ( corr ( calibration error , override rate ) = + 0.499 ). Clinicians override 43.3 % of model recommendations overall, but the override rate is significantly lower for well-calibrated predictions. Calibration quality explains ≈ 25 % of the variance in trust, confirming that trust is trajectory-dependent, not merely accuracy-dependent. See Figure 5.
 Claim 3
(Diversity–risk coupling). A decrease in D ^ is associated with an increase in R ^ after controlling for accuracy, validating Condition 3.
  • Test.
Regression of log R ^ on log D ^ with accuracy covariates. Result (Experiment 2, real LLMs + simulated): H2a and H2b supported. In real multi-model experiments (Ollama: Llama 3.1, Phi 4, Qwen 2.5, GPT-oss on n = 50 items), correlated error decreases super-linearly as judge count increases: power-law exponent b ≈ − 2.2 (H2b), confirming that monoculture risk accelerates faster than linear ensemble reduction predicts. The diversity–risk coupling (H2a) is validated: configurations with higher cross-model diversity exhibit lower correlated error (mean D = 0.14 – 0.18 across model pairs, correlated error < 0.04 for n ≥ 3 judges). Simulated multi-model experiments (63 configurations, β 1 = − 3.32 ) provide additional support at larger scale. See Figure 6.
 Claim 4
(Optimal verification depth). There exists a finite n * maximizing S ^ ( n ) .
  • Test.
Compute S ^ ( n ) for n ∈ { 0 , 1 , … , 10 } ; test unimodality. Result (Experiment 3, simulated & real LLMs): H3 supported. In the simulated graduated pipeline, S ^ ( n ) is unimodal with n * = 4 (committee vote layer): S ^ ( 0 ) = − 0.018 → S ^ ( 4 ) = + 0.063 → S ^ ( 10 ) = − 0.103 . In real-model experiments (Ollama: Llama 3.1, Phi 4, Qwen 2.5 on n = 50 items), the same pattern emerges: self-critique degrades accuracy from 88.0% to 56.0% (the self-critique dip), cross-model verification recovers accuracy to 90.0%, and further layers show diminishing returns. The stability functional peaks at n * = 2 (cross-model layer) with S ^ ( 1 ) = − 0.105 → S ^ ( 2 ) = − 0.098 → S ^ ( 4 ) = − 0.126 . Bootstrap 95% CIs ( B = 1000 ) are tight ( ≤ 0.01 width at the peak). See Figure 7.
 Claim 5
(Basin membership predicts trust). Trajectories classified as in- B * ^ yield higher mean T ¯ than out-of-basin trajectories, controlling for terminal accuracy.
  • Test.
Estimate B * ^ via the indicator Φ ^ (Definition 12); classify trajectories; compare T ¯ distributions with accuracy covariates. Result (Experiment 4, MIMIC-IV + simulated pool): H4 supported. On the MIMIC-IV cohort ( n = 3 , 000 , single-model accommodation D min = 0 ), in-basin trajectories at Φ * = 0.10 yield 85.6 % clinician trust versus 55.4 % for out-of-basin trajectories (Cohen’s d = 0.70 , trust gap + 0.30 ). On the pooled cross-experiment cohort ( n = 3 , 065 ), in-basin configurations achieve 80.3 % accuracy versus 56.4 % out-of-basin ( d = 0.69 ). One notable pattern is that single-model systems ( D = 0 ) fall below the diversity floor D min by construction, which under the proposed metric translates to elevated monoculture risk. However, this is a property of the metric, not a claim about the impossibility of trustworthy single-model systems in practice.

8.1. Baseline Comparison

Table 1. Baseline comparison from simulated experiments. Accuracy = majority-vote correctness; S ^ = stability functional (higher is better); D ^ = mean diversity; K ^ = cumulative cost per decision (USD). Single-model through temperature-ensemble values from Experiment 2 ( n = 4 judges, cross-model simulated pool); the low majority-vote accuracy reflects the presence of shared bias categories across models (trap items). Framework values from Experiment 3 at n * = 4 (graduated verification pipeline). MIMIC-IV model (Experiment 1): AUC = 0.773 , reported separately as it uses a different clinical prediction task.
Table 1. Baseline comparison from simulated experiments. Accuracy = majority-vote correctness; S ^ = stability functional (higher is better); D ^ = mean diversity; K ^ = cumulative cost per decision (USD). Single-model through temperature-ensemble values from Experiment 2 ( n = 4 judges, cross-model simulated pool); the low majority-vote accuracy reflects the presence of shared bias categories across models (trap items). Framework values from Experiment 3 at n * = 4 (graduated verification pipeline). MIMIC-IV model (Experiment 1): AUC = 0.773 , reported separately as it uses a different clinical prediction task.
Method Accuracy Stability S ^ Diversity D ^ Cost K ^
Single-model judge 0.554 − 0.030 0.000 $0.005
Majority-vote ensemble 0.356 − 0.018 0.485 $0.020
Temperature-sampled ensemble 0.438 − 0.018 0.434 $0.020
Trajectory framework (ours) 0.816 + 0 . 063 0.365 $0.049
Figure 5. Experiment 1 (MIMIC-IV, n = 3 , 000 test stays). (A) Reliability diagram: predicted mortality probability versus observed mortality rate. The logistic regression model (AUC = 0.773 , Brier = 0.188 ) is reasonably calibrated in the low-to-mid range but underpredicts at high risk. (B) Clinician trust (1−override rate) by confidence bin. Green bars indicate bins where calibration error ≤ δ = 0.1 ; red bars indicate miscalibrated bins. Well-calibrated bins consistently receive higher trust, supporting Claim 2 (H1).
Figure 5. Experiment 1 (MIMIC-IV, n = 3 , 000 test stays). (A) Reliability diagram: predicted mortality probability versus observed mortality rate. The logistic regression model (AUC = 0.773 , Brier = 0.188 ) is reasonably calibrated in the low-to-mid range but underpredicts at high risk. (B) Clinician trust (1−override rate) by confidence bin. Green bars indicate bins where calibration error ≤ δ = 0.1 ; red bars indicate miscalibrated bins. Well-calibrated bins consistently receive higher trust, supporting Claim 2 (H1).
Preprints 219974 g005
Figure 6. Experiment 2 (simulated multi-model judges, 63 configurations). (A) Diversity–risk scatter: each point is a configuration; color encodes majority-vote accuracy. The dashed red line shows the Condition 3 boundary R = R 0 ( 1 − D ) . Configurations above the boundary satisfy the diversity–risk coupling. (B) Correlated-error growth as judge count n decreases (cross-model: blue; same-model temperature-sampled: magenta). The power-law exponent b = − 1.94 confirms super-linear monoculture growth (H2b).
Figure 6. Experiment 2 (simulated multi-model judges, 63 configurations). (A) Diversity–risk scatter: each point is a configuration; color encodes majority-vote accuracy. The dashed red line shows the Condition 3 boundary R = R 0 ( 1 − D ) . Configurations above the boundary satisfy the diversity–risk coupling. (B) Correlated-error growth as judge count n decreases (cross-model: blue; same-model temperature-sampled: magenta). The power-law exponent b = − 1.94 confirms super-linear monoculture growth (H2b).
Preprints 219974 g006
Figure 7. Experiment 3 (simulated recursive verification, 500 items). (A) Stability functional S ^ ( n ) with bootstrap 95% CI ( B = 1000 ). The curve is unimodal with n * = 4 (committee vote layer), confirming Claim 4 (H3). The self-critique dip at n = 1 shows that same-model verification adds cost without benefit. (B) Error rate (red) and cumulative cost (orange) versus depth. Error decreases steeply through depth 3, then plateaus while cost grows linearly—the benefit/cost crossover at n * = 4 is the framework’s quantitative prediction.
Figure 7. Experiment 3 (simulated recursive verification, 500 items). (A) Stability functional S ^ ( n ) with bootstrap 95% CI ( B = 1000 ). The curve is unimodal with n * = 4 (committee vote layer), confirming Claim 4 (H3). The self-critique dip at n = 1 shows that same-model verification adds cost without benefit. (B) Error rate (red) and cumulative cost (orange) versus depth. Error decreases steeply through depth 3, then plateaus while cost grows linearly—the benefit/cost crossover at n * = 4 is the framework’s quantitative prediction.
Preprints 219974 g007

9. Discussion

The framework reframes AI evaluation as a dynamical-systems problem. Under this view, trust emerges not from static performance metrics but from trajectory-level properties: bounded risk drift, sustained calibration, preserved diversity, and bounded verification cost. This reframing unifies several currently disjoint research areas.
  • Calibration as a trajectory property.
Standard calibration research [7,9] studies the joint distribution of confidence and accuracy at a single time point. Our framework recovers this as Condition 1 that must hold along a trajectory, not only at its endpoint.
  • Ensemble diversity as a state coordinate.
The classical ambiguity decomposition [13] becomes a coordinate (D) of the state, dynamic and measurable per decision rather than fixed per model collection.
  • LLM-as-a-judge as operator composition.
LLM judges instantiate the J operator on a single-model trajectory and the A operator on an ensemble. Position and length biases documented by Wang et al. [12] manifest as systematic perturbations of J that, under repeated application, can drive Φ upward and trigger the collapse modes of Definition 11. The framework does not claim a one-to-one mapping between any single named bias and a single collapse mode; rather, it predicts that persistent bias increases Φ and thereby reduces basin membership, which Experiment 4 measures directly.
  • Recursive verification as cost–risk trade-off.
Verifier scaling [21] is recast as a trajectory in ( V , K , R ) , with Proposition 2 predicting the existence of a finite optimal depth. Experiment 3 confirms this prediction: S ^ ( n ) peaks at n * = 4 and declines monotonically thereafter, formalizing the “verification is the real cost” argument developed across the computational epistemics program [3] and the trust-budget control framework of Yu [6], a finding consistent with the verification tax identified in general AI auditing [30].
  • Multi-agent evaluation as collective epistemics.
The aggregation operator A and ensemble sub-trajectories (Definitions 7–8) instantiate the collective computational epistemics framework of Yu [5], formalizing how multi-agent networks produce diversity and correlated risk. Experiment 2 quantifies this: correlated error drops with diversity ( β 1 = − 3.3 ) and grows super-linearly under monoculture ( b = − 1.9 ).
  • Trust adoption as a trajectory outcome.
Scoping reviews of AI adoption barriers [2] and algorithmic fairness frameworks [31] identified structural barriers to clinician trust at the population level. The present framework supplies the missing mechanistic layer: adoption barriers are, formally, trajectories that exit B * due to calibration failure, monoculture, or verification breakdown. Experiment 4 confirms this directly: in-basin trajectories receive 85.6 % clinician trust versus 55.4 % for out-of-basin trajectories ( d = 0.70 ). Clinical AI governance frameworks [15,17] become operator sequences whose stability properties can now be analyzed rather than merely asserted.
  • Monoculture risk in single-model systems.
A pattern emerging from Experiment 4 is that single-model AI ( D = 0 ) falls below the diversity floor D min by construction under the proposed diversity metric. This is a property of the metric rather than a claim about fundamental instability: a single model augmented with temporal diversity, retrieval augmentation, or human oversight could plausibly satisfy the diversity condition through non-ensemble means. The metric’s implication is that institutional pipelines relying on a single evaluator should be scrutinized for monoculture risk, not that they are categorically unstable.
  • Implications for AI governance.
The framework provides concrete design principles for trustworthy AI pipelines. First, calibration alone is insufficient: a well-calibrated single model still violates the diversity floor and is classified as unstable. Second, the stability functional S gives a quantitative target for pipeline optimization: rather than maximizing accuracy alone, system designers should maximize S , which explicitly penalizes monoculture and over-verification. Third, the optimal verification depth n * is finite and measurable—resources spent beyond n * degrade rather than improve stability. These principles translate directly to regulatory standards: a minimum diversity requirement ( D ≥ D min ) and a maximum cost–risk threshold ( κ , η ) could be specified per domain, and compliance assessed via the Φ indicator.

10. Limitations

The framework relies on operational proxies (Definition 14) for latent variables; these proxies introduce measurement noise whose magnitude varies across domains. Trajectory estimation depends on system logging granularity: low-granularity logs (e.g., aggregate decision outcomes) yield coarser trajectories than per-token logs. The stability zone B * ^ is estimated via the indicator Φ ^ (Definition 12), whose boundary depends on the pre-registered threshold Φ * ; we mitigate threshold-selection bias by reporting sensitivity sweeps (Section 8). Experiments 2 and 3 combine simulated pipelines with real-model validation using local LLMs (Ollama: Llama 3.1, Phi 4, Qwen 2.5, GPT-oss). The real-model experiments confirm the predicted diversity–risk and self-critique effects, though with limited sample size ( n = 15 –50 items per configuration). Larger-scale validation with remote APIs (OpenAI, Anthropic, Google) would strengthen quantitative precision. The override definition in Experiment 1 (treatment–recommendation discrepancy) is an operational proxy for trust; richer signals from CPOE cancellation logs or NLP on clinical notes could refine the trust estimator. Finally, the cross-sectional H1 analysis (per-stay, first-24h) is a weaker test than the full temporal-trajectory version of Claim 2; multi-window trajectory extraction from MIMIC-IV is a natural extension.

11. Conclusions

We introduced a dynamical-systems formulation of AI evaluation in which trust is defined as a property of epistemic trajectories in a structured state space. The framework provides a unified coordinate system for AI systems, LLM judges, and human reviewers; a composable operator algebra over that space; a stability theory with four named collapse modes; a measurement model with empirical estimators for all state variables; and four experiments whose outcomes validate all five trajectory-level claims. On MIMIC-IV ICU data, calibrated model outputs receive 16.7 % more clinician trust than miscalibrated outputs (H1), and in-basin trajectories yield 85.6 % trust versus 55.4 % for out-of-basin trajectories with a moderate-to-large effect size ( d = 0.70 , H4). Multi-model experiments confirm that diversity reduces correlated risk super-linearly (H2, confirmed with both simulated and real LLMs). The stability functional is unimodal in verification depth with finite optima n * = 4 (simulated) and n * = 2 (real LLMs), confirming diminishing returns to verification (H3). The framework’s prediction that same-model self-verification degrades accuracy is confirmed in real models (88% to 56%). The eight state variables admit empirical estimators (Section 6), each claim has a falsifiable test (Section 8), and each collapse mode has a detector (Section 6). All code, data pipelines, and pre-registered configurations are released for reproducibility (see Data Availability).

Institutional Review Board Statement

This study uses the MIMIC-IV database, a de-identified public clinical dataset approved by the MIT Institutional Review Board (IRB) and the Beth Israel Deaconess Medical Center IRB. No additional IRB approval was required. Real LLM experiments used local models via Ollama on institutional hardware; no external API services were used for model inference.

Data Availability Statement

MIMIC-IV is available from PhysioNet (https://physionet.org/content/mimiciv/). The extracted feature dataset (10,000 ICU stays), all experimental code, pre-registered configurations, and analysis pipelines are available at https://github.com/yuyunguo/epistemic-state-space.

Acknowledgments

Portions of the experimental code and manuscript text were developed with assistance from AI language models (Claude, OpenAI). All scientific claims, experimental designs, analyses, and final editorial decisions are the sole responsibility of the author. The AI tools were used for code generation, editing, and literature synthesis under the author’s direct supervision.

Conflicts of Interest

The author declares no conflicts of interest.

Use of Artificial Intelligence

Please describe in detail any and all uses of artificial intelligence (AI) or AI-assisted tools used in the preparation of the manuscript. This may include, but is not limited to, language translation, language editing and grammar, or generating text. Alternatively, please state that “AI or AI-assisted tools were not used in drafting any aspect of this manuscript”.

Appendix A. Proof Sketches

  • Stability indicator convexity.
The stability indicator Φ (Definition 12) is convex in the state coordinates because each term max ( 0 , · ) is convex and the composition is a positive linear combination. This convexity implies via Jensen’s inequality that aggregation by weighted mean does not increase Φ beyond the mean of the individual evaluator values, which motivates the use of Φ as a diagnostic for ensemble stability.
  • Proposition 2 (optimal depth).
The finite-maximizer claim is established in body. The earlier (withdrawn) concavity claim would have yielded uniqueness of n * ; without it, n * is finite but need not be unique. Empirical Experiment 3 reports the realized S ( n ) curve and tests unimodality post hoc.

Appendix B. Extended Experiments

Completed sensitivity analyses: (i) judge count n ∈ { 1 , 2 , 4 , 8 } ; (ii) prompt perturbation (3 variants); (iii) sampling temperature τ samp ∈ { 0.0 , 0.3 , 0.7 } ; (iv) cross-model vs. same-model comparison; (v) bootstrap sample size B = 1000 . Additional sensitivity sweeps (e.g., n ∈ { 16 } , higher temperatures, alternative error-cost weights) are natural extensions.

Appendix C. Implementation Details

Trajectory extraction code, calibration and diversity metric implementations, and the recursive judge simulator are released with this manuscript. Pseudocode for the empirical trajectory estimator T ^ ( L ) :
for t in 0..T:
    C_hat[t] = confidence(model, x[t])
    T_hat[t] = 1 - override[t]
    U_hat[t] = variance(judges[t])
    V_hat[t] = count_verification_steps(log[t])
    R_hat[t] = error_prob[t] * harm_weight[x[t]]
    K_hat[t] = alpha_lat*latency[t] + alpha_cmp*cost[t]
              + alpha_hum*human_time[t]
    D_hat[t] = 1 - mean_pairwise_similarity(judges[t])
    X_hat[t] = grounded_tokens[t] / total_tokens[t]
return Trajectory(S_hat[0..T])

References

  1. Yu, Y.; Gomez-Cabello, C.A.; Haider, S.A.; Genovese, A.; Prabha, S.; Trabilsy, M.; Collaco, B.G.; Wood, N.G.; Bagaria, S.; Tao, C.; et al. Enhancing Clinician Trust in AI Diagnostics: A Dynamic Framework for Confidence Calibration and Transparency. Diagnostics 2025, 15, 2204. [Google Scholar] [CrossRef] [PubMed]
  2. Hassan, M.; Kushniruk, A.; Borycki, E. Barriers to and Facilitators of Artificial Intelligence Adoption in Health Care: Scoping Review. JMIR Hum. Factors 2024, 11, e48633. [Google Scholar] [CrossRef] [PubMed]
  3. Yu, Y. Computational Epistemics of Biological Discovery, Part I: The Real Cost of AI Isn’t Generation—It’s Verification. 2026. [Google Scholar] [CrossRef]
  4. Yu, Y. Computational Epistemics of Biological Discovery, Part II: The Epistemic Frontier, Technology Discontinuities, and the Friction of AI-Driven Discovery. 2026. [Google Scholar] [CrossRef]
  5. Yu, Y. Computational Epistemics of Biological Discovery, Part III: Collective Computational Epistemics and Multi-Agent Networks. 2026. [Google Scholar] [CrossRef]
  6. Yu, Y. Computational Epistemics of Biological Discovery, Part IV: Trust Budgets and the Optimal Control of Scientific Discovery. 2026. [Google Scholar] [CrossRef]
  7. Guo, C.; Pleiss, G.; Sun, Y.; Weinberger, K.Q. On Calibration of Modern Neural Networks. In Proceedings of the Proceedings of the 34th International Conference on Machine Learning (ICML), 2017. [Google Scholar]
  8. Naeini, M.P.; Cooper, G.F.; Hauskrecht, M. Obtaining Well Calibrated Probabilities Using Bayesian Binning. In Proceedings of the Proceedings of the AAAI Conference on Artificial Intelligence, 2015. [Google Scholar]
  9. Kumar, A.; Liang, P.S.; Ma, T. Verified Uncertainty Calibration. Advances in Neural Information Processing Systems (NeurIPS), 2019. [Google Scholar]
  10. Van Calster, B.; McLernon, D.J.; van Smeden, M.; Wynants, L.; Steyerberg, E.W. Calibration: the Achilles Heel of Predictive Analytics. BMC Med. 2019, 17, 230. [Google Scholar] [CrossRef] [PubMed]
  11. Zheng, L.; Chiang, W.L.; Sheng, Y.; et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2023. [Google Scholar]
  12. Wang, P.; Zheng, L.; Jiang, Z.; et al. Large Language Models are not Fair Evaluators. arXiv 2023, arXiv:2305.17926. [Google Scholar]
  13. Krogh, A.; Vedelsby, J. Neural Network Ensembles, Cross Validation, and Active Learning. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 1994. [Google Scholar]
  14. Wang, X.; Wei, J.; Schuurmans, D.; et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In Proceedings of the International Conference on Learning Representations (ICLR), 2023. [Google Scholar]
  15. Char, D.S.; Shah, N.H.; Magnus, D. Implementing Machine Learning in Health Care — Addressing Ethical Challenges. N. Engl. J. Med. 2018, 378, 981–983. [Google Scholar] [CrossRef] [PubMed]
  16. Li, J.; Zhou, Z.C.; Wang, Z.C.; Lv, H. Prioritizing human-AI collaboration in healthcare: the TRIAD framework for trustworthy governance, real-world, and integrated adaptive deployment. Mil. Med. Res. 2026, 12, 97. [Google Scholar] [CrossRef] [PubMed]
  17. Topol, E.J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 2019, 25, 44–56. [Google Scholar] [CrossRef] [PubMed]
  18. Kadavath, S.; Conerly, T.; Askell, A.; et al. Language Models (Mostly) Know What They Know. arXiv 2022, arXiv:2207.05221. [Google Scholar]
  19. Kuhn, L.; Gal, Y.; Farquhar, S. Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation. In Proceedings of the International Conference on Learning Representations (ICLR), 2023. [Google Scholar]
  20. Lin, S.; Hilton, J.; Evans, O. Teaching Large Language Models to Express Knowledge Uncertainty in Words. arXiv 2022, arXiv:2212.13691. [Google Scholar]
  21. Cobbe, K.; Kosaraju, V.; Bavarian, M.; et al. Training Verifiers to Solve Math Word Problems. arXiv 2021, arXiv:2110.14168. [Google Scholar]
  22. Johnson, A.E.W.; Bulgarelli, L.; Pollard, T.; et al. MIMIC-IV (version 1.0). PhysioNet 2021. [Google Scholar]
  23. Strogatz, S.H. Nonlinear Dynamics and Chaos, 2 ed.; CRC Press, 2018. [Google Scholar]
  24. Khalil, H.K. Nonlinear Systems, 3 ed.; Prentice Hall, 2002. [Google Scholar]
  25. Page, S.E. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies; Princeton University Press, 2008. [Google Scholar]
  26. Goldman, A.I. Knowledge in a Social World; Oxford University Press, 1999. [Google Scholar]
  27. Geifman, Y.; El-Yaniv, R. Selective Classification for Deep Neural Networks. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2017. [Google Scholar]
  28. Mozannar, H.; Sontag, D. Consistent Estimators for Learning to Defer to an Expert. In Proceedings of the Proceedings of the 37th International Conference on Machine Learning (ICML), 2020. [Google Scholar]
  29. Hüllermeier, E.; Waegeman, W. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods. Mach. Learn. 2021, 110, 457–506. [Google Scholar] [CrossRef]
  30. Wang, J.Z. The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime. 2026. [Google Scholar] [CrossRef] [PubMed]
  31. Chen, R.J.; Wang, J.J.; Williamson, D.F.K.; Chen, T.Y.; Lipkova, J.; Lu, M.Y.; Sahai, S.; Mahmood, F. Algorithmic Fairness in Artificial Intelligence for Medicine and Healthcare. Nat. Biomed. Eng. 2023, 7, 719–742. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.