Submitted:
13 November 2026
Posted:
16 November 2026
You are already at the latest version
Abstract
Triage is a constrained, high-frequency clinical decision: at the moment of first contact, a patient must be assigned an urgency level and a disposition lane under time pressure. Two recent lines of work address this decision from opposite directions. System-One decision models, exemplified by JEV and its open medical variant MedJev, answer typed questions (a choice among options, an ordinal score, or the probability of a binary event) about a patient state in a single forward pass, returning calibrated probabilities without generating text; they match or exceed supervised models on physiologic outcomes such as in-hospital mortality but underperform them on operational outcomes such as emergency department (ED) admission, which depend on institution-specific practice patterns that a zero-shot model cannot know. Patient similarity networks, in contrast, encode local practice implicitly: patients are nodes, edges are feature similarities, and graph neural networks (GNNs) over these networks classify triage acuity with high accuracy. Here we unify the two paradigms. We present SimJev, an open-source framework in which a patient similarity network built over a historical cohort supplies a local-practice prior to a System-One decision model through two mechanisms: (i) neighborhood-augmented states, in which the outcome summary of the \(k\) most similar historical patients is injected into the model's input state, and (ii) log-odds fusion, in which the model probability and the similarity-network probability are combined with a weight fit on 50–200 labeled local examples. The framework additionally implements the full GNN node-classification pipeline (GCN, GATv2, GraphSAGE) as a head-to-head comparator. On a public triage cohort of 6,552 patient encounters, the GraphSAGE arm outperforms the \(k\)-nearest-neighbor similarity prior (AUROC 0.984 vs. 0.932; AUPRC 0.686 vs. 0.366), and fusing the GNN probability with a decision-model probability improves calibration (ECE 0.028 to 0.007) without loss of discrimination. We then evaluate the framework end-to-end against a real, locally served open System-One backend (OpenJev 0.1 serving DiffusionGemma-26B) on the same cohort, with a solo-read protocol, two state encodings, and a field-attribution probe, and obtain four findings. First, the zero-shot admission probability is not merely uninformative but unstable in sign: the same frozen model, asked the same question about the same patients, is informative when two categorical fields are absent from its state (AUROC 0.653, 95% CI 0.567–0.737), anti-informative when those fields are present as arbitrary integer codes (0.397, 0.322–0.485), and chance-level when they are present as semantic strings (0.474, 0.386–0.549). Second, the probe localizes two mechanisms: co-asking questions in one request contaminates the answers — the admission question drops from 0.638 asked alone to 0.401 inside a three-question panel, consistent with joint denoising coupling the answer slots — and replacing integer codes with semantic strings removes the anti-informative signal (0.360 to 0.531 within run). Third, neighborhood-augmented states are robust across every configuration tested (AUROC 0.785–0.919), perform best under semantic encoding (0.915), and an echo-control ablation — each patient receiving a random other patient's neighborhood block — collapses performance to chance, establishing that the model genuinely conditions on block content. Fourth, log-odds fusion degrades gracefully in every regime: against anti-informative, contaminated, and chance-level model components the fitted weight falls to zero and the fused arm matches the prior (AUROC 0.917–0.919, ECE 0.004); we find no discrimination gain over the prior and report a borderline trend observed in one configuration (+0.022, \(p \approx 0.06\)) as unconfirmed. We state the scope of these claims explicitly: on a single public cohort, the framework demonstrates feasibility, calibration transfer, and a mechanism-of-action result — the similarity prior is the active ingredient of the augmented decision model — and, as its methodological contribution, that single-pass probability readouts require state-encoding, question-composition, and phrasing ablations as standard practice; discrimination headroom over the prior on credentialed MIMIC-IV-ED data remains the endpoint of the companion study. We release the framework, including the evaluation harness, the OpenJev wire-API integration, the field-attribution probe, the fine-tuning recipe, the synthetic-data mode, and all augmentation mechanisms, as open source.
Keywords:
patient triage
; System-One models
; decision models
; patient similarity networks
; graph neural networks
; clinical decision support
; calibration
; emergency department
1. Background
Patient triage is the gateway decision of emergency and inpatient care. At first contact, a clinician must convert a heterogeneous collection of observations — vital signs, chief complaint, demographics, prior utilization — into two coupled decisions: how urgently the patient needs to be seen, and where the patient should go (discharge, ward, step-down unit, or intensive care). The decision is constrained in a specific sense: the output space is small and fixed, the input is a structured state, and the cost of a delayed answer is measured in queueing time for every patient behind the one being triaged. Errors of under-triage and over-triage are both clinically consequential, and human triage is known to be subjective and variable across operators and shifts.
A growing literature applies machine learning to triage, with systematic reviews reporting promising discrimination but also flagging validation, calibration, and implementation gaps that have slowed adoption. Separately, the cognitive-science framing of triage as a fast, intuitive System 1 decision — as opposed to a slow, deliberative System 2 workup — has been examined empirically in the emergency department, where intuitive acuity prediction performs comparably to deliberative prediction for disposition but exhibits characteristic miscalibrations.
Two recent technical developments motivate the present work. The first is the emergence of System-One decision models. Unlike generative large language models, which produce an answer by autoregressive decoding and require parsing, a System-One model receives a state (text or JSON) together with a set of typed questions and returns a full probability distribution per question in a single forward pass. Jev, the first model of this class, defines three question types: Choice (pick one of up to 255 labeled options; returns a probability per option and a confidence summarizing how concentrated the spread is), Score (an ordinal scale with one description per level; returns a probability-weighted average level position), and Noul (the probability that a binary statement is true; the number is itself the confidence). Inference completes in tens to hundreds of milliseconds at a small fraction of the cost of generative inference, and the interface makes no side effects: all thresholds, gates, and actions live in the calling code. MedJev, an open System-One model released with a full evaluation harness, was benchmarked on ten medical tasks; it matched or exceeded supervised gradient-boosting models on in-hospital mortality (AUROC 0.926 vs. 0.902 on MIMIC-IV) and 30-day readmission, while underperforming supervised models on operational outcomes such as ED triage admission (AUROC 0.745 vs. 0.809). The same study showed that its zero-shot patient-level probabilities are miscalibrated with respect to local outcome prevalence (expected calibration error, ECE, of 0.286 for in-hospital mortality) and that a two-parameter recalibration on 50–200 labeled local examples reduces ECE to 0.007.
The second development is the patient similarity network approach to triage. In this paradigm, each patient is a node; edges connect patients whose observation vectors are similar under a chosen measure (cosine similarity, Euclidean, Manhattan, or Minkowski distance) above or below a threshold; and the resulting network is embedded and classified with a graph neural network. Defilippo and colleagues [4,5] demonstrated this pipeline for emergency-code assignment, evaluating four similarity measures, multiple thresholds, and three GNN architectures (GCN, GATv2, GraphSAGE), and reported accuracy exceeding tabular baselines. The essential observation is that explicit modeling of similarity between patients captures hidden structure that per-patient tabular features do not: even when a single patient’s data are sparse, the outcomes of similar historical patients are informative.
These two paradigms have complementary failure modes, and the complementarity is not accidental. A zero-shot System-One model carries broad medical knowledge but no knowledge of the deployment institution: it cannot know that this ED admits elderly night-time arrivals more readily than its peer institutions, or that step-down capacity on the weekend shifts the admission threshold. Operational outcomes are precisely the outcomes governed by such local practice patterns. A patient similarity network, conversely, encodes local practice implicitly and exhaustively — but it is only as good as its historical cohort, it generalizes poorly to distribution shift, and it carries no general medical knowledge about physiology or disease. The gap where MedJev underperforms (operational outcomes) is exactly the gap a similarity prior should close; the gap where a similarity network is fragile (shift, sparse cohorts, physiologic reasoning) is exactly where a System-One model is strong.
In this preprint we make three contributions. First, we define and release simjev, an open-source framework that implements the Jev request interface (typed Choice/Score/Noul questions over JSON states), patient similarity networks with four similarity measures and thresholded graph construction, inductive k-nearest-neighbor retrieval with outcome summaries, GNN node classification over the thresholded network, and two mechanisms for combining the paradigms. Second, we formalize the two augmentation mechanisms — neighborhood-augmented states and log-odds fusion — and derive their properties, including graceful degradation when the similarity prior is uninformative. Third, we report a head-to-head evaluation on a public triage cohort comparing the k-nearest-neighbor similarity prior against the full GNN pipeline, together with a real-backend evaluation of both augmentation mechanisms — zero-shot, augmented, and fused System-One performance, a question-phrasing ablation, and an echo-control ablation — and a calibration analysis of the fused model. We deliberately scope this preprint to the framework and the public-data evaluation; a companion study will report System-One performance on credentialed MIMIC-IV-ED data, external and temporal validation, and a prospective shadow deployment.
2. Methods
2.1. Notation and Problem Statement
Let a patient encounter at prediction time be described by a feature vector comprising all information available at (triage vital signs, demographics, chief complaint, and, for inpatient tasks, the serialized event history in the window ). Let a historical cohort contain encounters with observed outcomes, where denotes a binary operational outcome (e.g., hospital admission decided at triage) and additional outcomes (ICU admission within 24 h, in-hospital mortality) may be carried in parallel. The triage task is to produce, for a new patient x, a set of calibrated probabilities over typed questions — binary outcomes, disposition choices, and urgency levels — such that downstream decision policies can act on them with explicit thresholds. We emphasize that the framework’s output is a probability vector, never an action; the mapping from probabilities to actions belongs to a decision policy owned by the deploying institution (Section 2.9).
2.2. Framework Overview
simjev is organized in five layers, mirroring the data flow from the electronic health record to the decision policy (Figure 1):
- State serialization. The patient’s available information is serialized into a compact JSON state. The state is the sole input channel to the decision model; everything the model knows about the patient must be explicit in it.
- Patient similarity index. A historical cohort is indexed by feature similarity (four measures, inductive retrieval). The index supplies two artifacts: a neighborhood outcome summary for any new patient (used by Mechanism A) and a local-practice prior probability (used by Mechanism B).
- System-One inference. A Jev-compatible backend — a locally served open System-One model, a hosted endpoint, or a labeled fallback for testing — answers a panel of typed questions about the (optionally augmented) state in a single pass.
- Fusion and recalibration. Model probabilities are recalibrated to local prevalence and, optionally, fused with the similarity prior (Mechanism B).
- Decision policy. Thresholds, confidence gates, and side effects are applied in institution-owned code, with full logging of the probability vector for audit.
The layering is deliberate: the similarity index and the decision model are independently auditable, each mechanism can be ablated without touching the others, and the entire pipeline runs on a laptop-class machine without data leaving the deployment site.
2.3. The System-One Request Interface
A Jev request is a pair where s is a state (text or JSON) and is a set of typed questions. Each question has one of three types:
- Choice: a question q over an option set , , optionally with a one-line description per option. The answer is a distribution over the options, a selected option , and a confidence summarizing the concentration of the distribution.
- Score: a question over an ordinal scale with levels , each described by one sentence. The answer is the probability-weighted average level position , which distinguishes a patient sitting between two levels (e.g., ) from one firmly at a level.
- Noul: a question whose answer is the probability that the statement q is true of the state. No separate confidence is returned; the probability is the confidence.
All questions in a request are answered in a single forward pass; there is no autoregressive generation and no answer parsing. Two properties of this interface are load-bearing for the framework. First, extensibility without cost: adding questions to the panel does not add forward passes, so a deployment can ask a broad panel (deterioration, admission, disposition, urgency) and let the policy layer decide which readings matter. Second, state transparency: because the state is an explicit JSON document, augmentation mechanisms are ordinary edits to the document rather than changes to the model, which is what makes the similarity prior injectable into a frozen model (Section 2.5).
The triage question panel used throughout this work is: a Choice over disposition lanes with one-line clinical descriptions; a Score over urgency levels ; and one Noul per binary outcome of interest (admission; optionally ICU utilization within 24 h and in-hospital mortality).
2.4. Patient Similarity Networks
2.4.1. Network Construction
Following Defilippo et al., each historical patient is a node, and the network is constructed by thresholding a pairwise similarity. Features are preprocessed by median imputation of missing values followed by min–max scaling, with all statistics computed on the historical cohort only. Four similarity measures are supported: cosine similarity , and the Euclidean, Manhattan, and Minkowski (order p) distances. An edge is created when for similarity, or for distances, where is a threshold parameter. The published pipeline evaluates a grid of thresholds per measure and reports strong threshold sensitivity: for cosine similarity, intermediate thresholds (0.94–0.95) were optimal, with lower thresholds producing near-complete graphs and degraded performance. Our implementation exposes the same grid and, for memory control, caps the neighbor degree at 200 per node during construction; the resulting graph is emitted in a format directly consumable by PyTorch Geometric.
2.4.2. Inductive k-Nearest-Neighbor Index
For the augmentation mechanisms we do not require the full network at inference time, only the neighborhood of a query patient. We therefore maintain an inductive k-nearest-neighbor index over the preprocessed cohort. Given a query x, the index returns the k nearest historical patients with their similarities (cosine similarity, or for distance metrics), and two derived artifacts:
the similarity-weighted local outcome rate, which serves as the Mechanism B prior; and a structured summary of the neighborhood (the count and rate of each outcome, plus aggregate statistics), which serves as the Mechanism A state block. Because the index stores only training-cohort statistics, retrieval for a new patient is strictly out-of-sample; no graph recomputation is needed when new patients arrive, which is the practical meaning of induction in this setting.
2.5. Augmentation Mechanisms
We formalize two mechanisms by which the similarity network informs the System-One model. They differ in where the combination happens: inside the model’s input (Mechanism A) or outside it, at the probability level (Mechanism B).
2.5.1. Mechanism A: Neighborhood-Augmented States
The state is augmented with a block summarizing the retrieved neighborhood:
where and are the count and rate of outcome o among the k nearest historical patients. The System-One model then answers its typed questions conditioned on . The mechanism requires no retraining and no access to model internals — the augmentation is an ordinary field of the state document — which preserves the zero-shot deployment model and the single-pass latency. Its risk is equally clear: a frozen model was never trained to condition on neighborhood blocks, and may ignore them or weight them unpredictably. We therefore treat Mechanism A as an empirical question and pair it with Mechanism B, which cannot fail silently.
2.5.2. Mechanism B: Log-Odds Fusion
Let be the System-One probability for a binary outcome (after local recalibration, Section 2.5.3) and the similarity prior of Eq. (1). We combine them as a log-linear opinion pool:
where is the logistic function and is a single mixing weight fit by minimizing the Brier score over a labeled tuning set of m examples, using bounded scalar optimization. The same –200 labeled examples required for local recalibration suffice, so fusion adds no labeling burden. Three properties motivate this form. Graceful degradation: if the prior is uninformative, the optimum moves to and the fusion reduces to the System-One model alone; if the model is uninformative, and the fusion reduces to the prior. Auditability: w is a single interpretable number, and both inputs to the fusion are stored per patient. Calibration transfer: because the pool operates on logits, a well-calibrated component contributes its certainty geometrically rather than being dominated by the other component’s scale. The same construction applies verbatim with the GNN probability substituted for the kNN prior, which defines the jev_fused_gnn arm of the evaluation.
2.5.3. Local Recalibration
Zero-shot patient-level probabilities do not account for local outcome prevalence. Following the MedJev methodology, we recalibrate the System-One output to the deployment population using a lightweight two-parameter affine map in logit space fit on the labeled tuning set. Recalibration is a prerequisite, not an option: in the published evaluation it reduced in-hospital mortality ECE from 0.286 to 0.007 using 50–200 labeled examples. In simjev the recalibration and fusion stages share the same tuning set, and the label-efficiency of the combined pipeline can be profiled with the bundled harness.
2.6. GNN Node-Classification Arm
To compare the augmentation approach against the full published pipeline, simjev implements the Defilippo et al. GNN arm end-to-end. The thresholded similarity network (Section 2.5 construction above) is embedded by a two-layer graph convolutional encoder with a linear head:
where E is the edge set and Conv is either GCNConv, GATv2Conv, or SAGEConv. The model is trained with a class-imbalance-weighted binary cross-entropy on the training nodes (positive weight , capped at 10 for optimization stability), Adam with weight decay , dropout , and model selection by validation AUROC on the tuning nodes. The split is inductive at the node level: the GNN observes the features of all nodes (transductive features) but the labels of training nodes only, mirroring the deployment situation in which historical outcomes are labeled and current patients are not. Predictions for tuning and test nodes are returned re-aligned to the caller’s index order; we note in Section 2.13 that this alignment is a nontrivial correctness point in any mask-based GNN evaluation.
2.7. Datasets
iatriage / Patient Priority Classification. A public cohort of 6,962 patient admission records with 16 features (age, gender, chest pain type, blood pressure, cholesterol, max heart rate, exercise angina, plasma glucose, skin thickness, insulin, BMI, diabetes pedigree, hypertension, heart disease, residence type, smoking status) and a four-level triage target {red, orange, yellow, green}, distributed with the iatriage repository (GPL-3.0) and originating from the Kaggle Patient Priority Classification dataset; this is the same cohort used in the Defilippo et al. evaluation. We preprocess it as in the published pipeline — removal of records with missing target, replacement of Unknown with missing values, mode imputation of categorical features, label encoding — with one deliberate deviation: we do not apply SMOTE or SMOTEENN resampling. Resampling must never touch the test split, and our primary metrics are threshold-free (AUROC, AUPRC), so imbalance is handled by reporting AUPRC alongside AUROC and by class-weighting inside the GNN loss. The binary outcome for the head-to-head evaluation is (immediate attention), with a prevalence of 1.8% in the analysis split.
A reproducibility hazard in this loader turned out to be scientifically informative and is documented here because it affects any pandas ≥ 3 reanalysis. The original loader detected categorical columns with a dtype == object check; pandas 3 introduced a dedicated str dtype, so residence_type and smoking_status silently failed the check and were dropped by the downstream numeric filter, leaving 14 features in the state with no warning, no error, and plausible metrics throughout. The defect was surfaced only by the field-attribution probe (Section 2.12), whose per-field inventory exposed the two missing columns. The fix (accepting any non-numeric, non-boolean column) restores all 16 features, and all primary real-backend results in this preprint use them. Because the accidental 14-feature configuration had already been evaluated end-to-end, we retain it as a discovery arm.
MIMIC-IV-ED (companion study). The framework includes a loader for the MIMIC-IV-ED triage tables (vital signs, pain, acuity) with admission outcomes derived from ED dispositions, matching the ED Triage Admission task of the MedJev evaluation. Because these data are credentialed, the loader is intended for local execution; we report no MIMIC results in this preprint.
Synthetic cohort. A self-contained generator produces triage-like cohorts in which a latent severity drives both vital signs and a physiologic admission component, while a site-specific practice pattern (a smooth function of age and time of day) drives an independent operational component. This structure — admission partially decoupled from physiology — is the exact regime in which the similarity prior is hypothesized to add value over physiology-only zero-shot models, and it allows the full pipeline to be exercised on any laptop without credentialed data.
2.8. Evaluation Protocol
All arms are compared on identical splits of each cohort: a training split for the similarity index and GNN, a tuning split (15%) for recalibration and fusion-weight fitting, and a held-out test split (30%). We report AUROC and AUPRC with 95% bootstrap confidence intervals (500 resamples), ECE with ten equal-width bins, and Brier score. The arms are: (i) knn_only, the similarity prior of Eq. (1) alone; (ii) jev_zeroshot, the System-One model on the plain state; (iii) jev_augmented, Mechanism A; (iv) jev_fused, Mechanism B with the kNN prior; (v) gnn_<model>, the GNN node-classification arm; and (vi) jev_fused_gnn, Mechanism B with the GNN probability. Ablation over the similarity measure, the neighborhood size k, and the graph threshold is supported through the command-line interface. Two protocol rules are enforced by construction rather than by discipline: imputation and scaling statistics are computed on the training split only, and no resampling is applied to any split.
2.9. Decision Policy
The framework’s contract is that the model never acts. A reference policy, included for illustration, implements three rules adapted from the System-One literature: a confidence gate that routes patients to a human triage officer when the Choice confidence falls below a floor (a flat option distribution usually indicates a patient who fits two lanes); conservative thresholds on Noul outputs for irreversible actions such as ICU admission or de-escalation; and complete logging of the probability vector, the neighborhood block, and the fusion inputs for every encounter. Threshold values are institution-specific parameters, not framework constants, and should be moved only once labeled local traffic exists.
2.10. Implementation
simjev is implemented in Python (NumPy, pandas, scikit-learn, SciPy; PyTorch and PyTorch Geometric for the GNN arm) and released under an MIT license at [repository URL]. The Jev interface supports three backends: an adapter for the OpenJev server’s wire API (POST /v1/systemone with typed noul/choice/score questions), which runs an Apache-2.0 diffusion model locally on Apple silicon or NVIDIA GPUs and keeps credentialed data on the deployment machine; a generic Jev-compatible HTTP client; and a clearly labeled heuristic fallback that exists only to exercise the plumbing end-to-end and must not be used clinically. A bundled recipe covers the Mac deployment path end-to-end: zero-shot serving of OpenJev, and — since OpenJev ships no training harness — an optional MedJev-style fine-tuning path in which a permissively licensed base model is LoRA-trained on the ED cohort to emit the System-One answer tokens and served behind the same wire API with a single-pass probability readout. The full pipeline, including the GNN arm, runs on a laptop; every model output is cacheable keyed by the exact request. The repository includes the iatriage dataset, the synthetic generator, the ablation sweep, and a test suite covering the similarity index, the fusion weight recovery, the metrics, the request format, the state augmentation, and the GNN alignment.
2.11. Real-Backend Evaluation Protocol
All System-One arms in the Results were served by OpenJev 0.1 [1], an open-source Jev-compatible decision server, running the DiffusionGemma-26B-A4B model (4-bit MLX quantization) locally on an Apple M4 Pro with 24 GB of unified memory; the evaluation campaign comprises approximately 30,000 single-pass requests across four full-cohort runs, shuffle ablations, and probe campaigns, at approximately one request per second. The Noul readout returns read directly from the model’s answer-token distribution in a single read-only pass; answer slots with entropy above 0.1 are re-read with fresh noise and averaged over four passes by the server.
Solo-read protocol. In the primary configuration each Noul question is asked in its own request, containing only the admission question. This protocol was adopted after we found that co-asking questions in a single request contaminates their answers (Section 2.15); the earlier three-question panel configuration (disposition Choice, urgency Score, admission Noul in one request) is retained and reported as a distinct experimental condition, because question-panel composition proved to be a first-class experimental variable that cuts in opposite directions for the zero-shot and augmented arms.
State encodings. Each full-cohort run was performed under two state encodings that differ only in how categorical fields are written into the JSON state: integer codes (the label encoding of the published pipeline, e.g. residence_type: 1) and semantic strings (e.g. residence_type: "Urban"). Numeric features are identical under both encodings.
Ablations and statistics. The phrasing ablation asks the zero-shot arm three progressively simpler phrasings (a neighborhood-presupposing phrasing, a statement-verification phrasing, and the plain interrogative). The echo-control ablation runs the augmented arm with each patient receiving a randomly reassigned neighborhood block, drawn by permuting the block-to-patient assignment with a fixed seed; the permutation preserves the marginal distribution of blocks while breaking the patient–neighbor correspondence, so any residual discrimination must come from the block’s format or marginal statistics rather than its content. Paired comparisons between arms use a paired bootstrap over test patients (2,000 resamples, fixed seed).
2.12. A Field-Attribution Probe for Single-Pass Readouts
To attribute a zero-shot readout’s behavior to specific state fields and request compositions, we implemented a probe that evaluates the same Noul question about the same stratified patient sample (all test positives plus an equal-fixed number of negatives, ) under controlled conditions: full (the complete state, question asked alone), panel (the same question co-asked inside the multi-question panel), singleton (each field alone in the state), dose (each field swept across its observed range with the remainder fixed at cohort medians), and semantic (categorical fields written as strings). Discrimination is computed as a rank-based AUROC with bootstrap confidence intervals. The probe is diagnostic, not evaluative: its sample is stratified rather than representative, and its per-field readouts inherit the server’s read noise (singleton AUROCs were observed to vary by up to 0.3 between identical runs on uncertain slots), so we use it to localize mechanisms — which fields, which compositions — and not to estimate performance.
2.13. A Methodological Pitfall: Label Alignment in Mask-Based GNN Evaluation
During verification we identified a silent correctness hazard in mask-based GNN evaluation that we document here because it affects any reproduction of the published pipeline. Boolean-mask extraction of predictions returns nodes in sorted node-index order, whereas the label vector indexed by a random permutation slice (y[i_test]) is in permutation order. Computing a validation metric between the two silently shuffles labels against predictions; the metric remains finite and plausible, training proceeds normally, and the only symptom is validation performance pinned near chance with early stopping selecting noise epochs. In our runs, correcting the alignment moved the GraphSAGE validation AUROC from approximately 0.50 to 0.99 on the iatriage cohort without any change to the model or the data. The same hazard arises when predictions are re-aligned to the caller’s index order, which requires the inverse permutation (a scatter by sorted positions), not a single argsort. We recommend that any mask-based GNN evaluation include an explicit alignment regression test; one is included in the released test suite.
2.14. A Second Pitfall: Question Phrasing Drives Sign-Flipped Zero-Shot Answers
The Noul readout is a single masked-token probability, and we found it acutely sensitive to question framing — sensitive enough to leave the model anti-informative under framings that look reasonable. Asked a plain interrogative about a raw JSON patient state, the zero-shot model returns answers that rank admitted patients below non-admitted ones (Table 2). An early version of our zero-shot arm reused the augmented arm’s phrasing, which presupposes context the state does not contain (“consistent with the similar historical patients shown”); a statement-verification framing (“Is it true that …”) reproduces the effect. None of these failure modes is visible in the metrics of a single run: every phrasing produces finite, plausible, confidently wrong probabilities, and the failure would have propagated silently into every downstream fusion had we not run the ablation. We document the effect because it generalizes: a single-pass probability readout inherits the prompt fragility of the underlying model without the safeguard of an output parser, and we recommend that System-One evaluations report a phrasing ablation alongside the main result. All phrasings and their per-arm assignments are fixed in the released question-set builder.
2.15. A Third Pitfall: Cross-Question Contamination in Jointly Denoised Answer Slots
The OpenJev backend denoises all answer slots of a request jointly: the answer tokens for co-asked questions are refined together, so each question’s answer distribution is conditioned on the other questions being answered in the same pass. We found this couples the answers to a degree that flips conclusions. Asked alone about a fixed patient state, the admission Noul question was mildly informative (probe AUROC 0.638, 95% CI 0.525–0.743); the identical question about the identical states, co-asked inside a three-question panel with a disposition Choice and an urgency Score, was anti-informative (0.401, 0.288–0.515), matching the full-cohort panel run (0.336). The solo answers were uniformly low-probability, whereas the panel answers saturated near 1.0 (panel ECE 0.760) — the co-asked questions, not the state, drove the response distribution. The upstream documentation warns that Score questions can ignore the state; our result shows the interference also runs through the jointly denoised slots of neighboring questions. We recommend that System-One evaluations report each question both alone and in its deployment composition; we return to this in Section 3.2, where panel composition cuts in opposite directions for the zero-shot and augmented arms.
2.16. Ethical and Governance Considerations
Triage is a resource-allocation decision with direct patient consequences, and its automation is governed by established ethical frameworks for critical care rationing, which require that prioritization remain grounded in individualized clinical assessment and proportionality. The framework is designed to those constraints: the model returns probabilities, never actions; a human triage officer owns every decision; confidence gates route uncertain cases to humans; and the neighborhood block provides case-based explanations (“7 of 10 similar historical patients were admitted”) that are more interpretable than either model alone. The open System-One models evaluated in this line of work are research artifacts, not regulated clinical devices, and weight licenses (CC BY-NC for MedJev) must be respected; for clinical deployment, the same readout pattern applies to permissively licensed base models. No patient-identifiable data were processed outside credentialed environments in this study.
3. Results
3.1. Head-to-Head Comparison on the Iatriage Cohort
Table 1 reports the held-out comparison on the iatriage cohort (6,552 analyzed encounters; 3,605 train / 982 tune / 1,965 test; outcome: red triage, prevalence 1.8%). The full GNN pipeline outperforms the kNN similarity prior on both discrimination metrics, with GraphSAGE the strongest architecture (AUROC 0.984, AUPRC 0.686 vs. 0.932 and 0.366 for the kNN prior), reproducing the published finding that explicit network modeling of patient similarity adds value over neighborhood voting. GCN and GATv2 perform comparably to each other and below GraphSAGE.
3.2. System-One Evaluation on a Real Backend
Table 2 reports the full-cohort evaluation against the locally served OpenJev backend (Section 2.11) under the solo-read protocol, for both state encodings; Table 3 collects the zero-shot and augmented arms across all evaluated configurations, including the earlier panel and 14-feature runs.
Four findings stand out.
(1) The zero-shot readout is unstable in sign. The same frozen model, asked the same admission question about the same patients, is informative in one configuration (0.653), anti-informative in another (0.397, 95% CI excluding 0.5 on the wrong side), and chance-level in a third (0.474). The instability is driven by the state representation and the question composition, not by the model’s medical knowledge: no configuration’s metrics look anomalous in isolation, and every configuration produces finite, plausible, well-calibrated-looking probabilities (solo-read ECE ). A single-configuration evaluation would have reported any of the three outcomes with full confidence.
(2) Two mechanisms, localized by the probe. First, cross-question contamination (Section 2.15): the admission question drops from 0.638 asked alone to 0.401 inside the three-question panel — identical states, identical question text — and the full-cohort runs reproduce the contrast (0.653 solo vs. 0.336 panel). Second, encoding sensitivity: arbitrary integer codes for categorical fields drive the anti-informative behavior. Within a single probe run on identical patients, switching the two restored categorical fields from codes to semantic strings moved the full-state readout from 0.360 to 0.531, and the full-cohort semantic run confirms chance-level rather than anti-informative behavior (0.474). The dose–response component of the probe shows the clinically backwards gradients that the codes induce (e.g. hypertension coded 1 reads as a discharge signal); no single field reproduces the full-state inversion, which is a joint-representation effect. Notably, the accidental 14-feature configuration created by the pandas-3 loader defect (Section 2.11) acted as a natural experiment: it is precisely the state without the two poisonous code fields, and it is the only configuration in which the zero-shot readout was informative.
(3) Neighborhood-augmented states are robust; semantic encoding is best. Across all four full-cohort configurations the augmented arm never fell below 0.785 and reached 0.915 under semantic encoding (AUPRC 0.216), statistically indistinguishable from the kNN prior itself (0.932). The echo-control ablation establishes that the arm reads block content: reassigning each patient a random wrong neighborhood block collapses the augmented arm from 0.889 to 0.368 in the panel configuration (95% CI 0.272–0.478, smoke-scale runs, indistinguishable from the zero-shot level in the same runs) and to 0.451 in the solo configuration, against 0.785–0.915 for correctly matched blocks. The panel–solo contrast also cuts the other way for this arm: the co-asked disposition question helps the augmented arm (0.919 panel vs. 0.785 solo, 14-feature codes), plausibly because the disposition answer anchors the jointly denoised admission answer to the block’s admission rate — the same coupling that poisons the zero-shot arm stabilizes the anchored one. Question-panel composition is therefore not a nuisance parameter but a design variable.
(4) Fusion degrades gracefully; no discrimination gain over the prior. In both final configurations the fitted weight fell to zero ( and ) and the fused arm matched the prior (AUROC 0.919 and 0.917 vs. 0.932; ECE 0.004). Across the campaign the fusion confronted an anti-informative component (this section), a contaminated panel component (), and a chance-level component (the echo-control run, , fused 0.910 vs. prior 0.909), and matched the better component in every case. We report for completeness that in the 14-feature solo configuration the fused arm showed a borderline AUROC gain over the prior (paired bootstrap , 95% CI , ) with a fusion weight of ; because the prior’s probabilities take only 218 distinct values on 1,965 test patients, this gain is attributable to tie-breaking by the model’s small log-odds contribution within the prior’s tie groups, and it did not replicate in the 16-feature configurations. We therefore claim calibration transfer and graceful degradation, not discrimination gain.
3.3. Fusion Improves Calibration Without Sacrificing Discrimination
Fusing the GraphSAGE probability with the System-One probability under Eq. (3) left discrimination unchanged (AUROC 0.984; Figure 2, left) and reduced ECE from 0.028 to 0.007 and Brier score from 0.024 to 0.010, with the fitted mixing weight allocating substantial weight to the decision-model component (Figure 2, right). The same behavior was observed for GCN (ECE 0.063 to 0.013) and GATv2 (0.037 to 0.005). This is the calibration-transfer property of the log-odds pool operating as designed, and it matters operationally: triage policies act on thresholds, so a discrimination gain that arrives with miscalibration is not usable, whereas the fused model’s probabilities can be thresholded directly.
3.4. Graceful Degradation of the Fusion
The fusion’s failure behavior was verified in two controlled regimes. With an uninformative decision-model component (the labeled heuristic backend on the synthetic cohort), the fitted weight moved to and the fused arm matched the prior alone; with an informative decision-model component and an uninformative prior (a controlled test in which the prior was replaced by noise), the weight moved to and the fusion recovered the model’s discrimination. The real-backend campaign then confronted the fusion with the pathological cases in the wild — an anti-informative component (16-feature codes, Table 2), a contaminated panel component (), and a chance-level component (the echo-control run, , fused 0.910 vs. prior 0.909 with identical ECE) — and in every case the fitted weight fell to zero or near zero and the fused arm preserved the prior’s discrimination (0.917–0.919 vs. 0.932) and calibration (ECE 0.004). The fusion therefore degrades to the better component in both directions, which is the property that makes it safe to deploy while the System-One backend is still being validated locally. The corollary is equally important: on this cohort the fusion adds nothing over the prior, and the borderline tie-breaking trend of Section 3.2 should not be read as headroom.
3.5. Scope of the Reported Results
We report these results with an explicit scope statement. The real-backend System-One results (Table 2 and Table 3) come from a single public cohort with a 1.8% positive rate, a single random split, and a binary red-triage outcome; the zero-shot instabilities are a property of one open backend model (DiffusionGemma-26B via OpenJev 0.1), and we do not claim they generalize to other System-One models or other outcomes. The probe’s per-field readouts inherit the server’s read noise on uncertain slots (singleton AUROCs varied by up to 0.3 between identical runs), so we advance only the within-run contrasts (codes vs. strings; alone vs. panel) as mechanistic evidence, not the per-field point estimates. On this cohort the prior, not the model, is the active ingredient: the augmented arm matches but does not exceed the kNN prior’s AUROC in any configuration, the fusion adds no discrimination, and the borderline tie-breaking trend is reported as unconfirmed. Whether a frozen System-One model can add discrimination beyond a similarity prior — the headroom question that motivates the framework — requires cohorts where the prior is weaker (credentialed MIMIC-IV-ED, operational admission outcomes, label-efficiency regimes) and is the subject of the companion study. The GNN comparison of Table 1 retains the scope reported above: it is a comparison between two similarity-based arms, and the AUPRC differences should be read against the prevalence.
4. Discussion
This preprint contributes a framework together with a real-backend evaluation, and its claims should be read accordingly. The central design claim is that the System-One interface and patient similarity networks are natural complements: the typed-question interface makes the similarity prior injectable as an ordinary state field or as a probability-level pool, and the similarity network supplies exactly the local-practice knowledge that zero-shot decision models lack on operational outcomes. The evaluation supports a sharpened version of that claim. The same frozen backend model that fails zero-shot — with a sign that depends on the state encoding and the question composition — conditions on the injected neighborhood evidence to reach prior-level discrimination in every configuration tested (0.785–0.919, best 0.915 under semantic encoding), the echo control shows the effect is carried by the block content rather than its format, and the fusion is safe against every pathological component it confronted (; fused ECE 0.004). On this cohort the similarity prior is the active ingredient of the augmented decision model.
Several limitations bound the conclusions. First, the reported results come from a single public cohort with a low positive prevalence and a four-level target reduced to a binary red-triage outcome, analyzed on a single random split; the published pipeline’s four-class formulation and SMOTE-balanced training differ from our imbalance-aware protocol, so our numbers are not directly comparable to the published accuracies. Second, the real-backend evaluation answers the framework’s motivating empirical questions on one cohort, but it also shows that on this cohort the prior, not the model, is the active ingredient: the augmented arm matches but does not exceed the kNN prior’s AUROC, and the fusion adds no discrimination. Whether a frozen System-One model can add discrimination beyond a similarity prior — in the 50–200 labeled-example regime where supervised models are starved, or on operational admission outcomes in credentialed MIMIC-IV-ED — remains the companion study’s primary endpoint. Third, the zero-shot instabilities are single-model observations; establishing them across System-One backends, and testing the contamination mechanism against backends that answer questions independently rather than by joint denoising, is a precondition for general claims. Fourth, the similarity prior inherits the fragility of any historically trained model: it encodes the practice patterns of the cohort, and under surge conditions or deliberate practice change those patterns are stale; the fusion weight provides a single, monitorable knob for down-weighting the prior when drift is detected, but drift detection itself is not yet implemented. Fifth, the calibration results are in-sample for the fusion weight (fit on the tuning split, evaluated on the test split) and should be confirmed under temporal validation before any deployment claim.
Four methodological observations generalize beyond this framework, and together they constitute, we believe, the preprint’s most transferable contribution. The label-alignment pitfall of Section 2.13 silently produces chance-level GNN results in mask-based evaluations; we recommend an alignment regression test as standard practice. The phrasing pitfall of Section 2.14 shows that a single-pass probability readout can be driven sign-flipped by framing alone. The contamination pitfall of Section 2.15 shows that co-asked questions are not independent measurements: in a jointly denoised System-One backend, question-panel composition is an experimental variable that must be reported, and it can cut in opposite directions for different arms (poisoning the zero-shot readout, stabilizing the anchored one). The encoding pitfall of Section 3.2 shows that arbitrary integer codes — the default output of every label-encoding pipeline — can drive a single-pass readout clinically backwards, and that semantic strings are the safer state representation. None of these failure modes is visible in the metrics of a single run; each was caught only by an ablation or a probe, which is our argument for treating encoding, composition, and phrasing ablations as standard practice in System-One evaluation. Finally, the calibration-first reporting standard adopted here — AUROC and AUPRC alongside ECE and Brier for every arm — reflects the reality that triage policies consume thresholds, so a model that cannot be thresholded safely is not a triage model regardless of its discrimination.
The roadmap toward the companion study is concrete: zero-shot, augmented, and fused System-One evaluation on MIMIC-IV-ED with a locally served OpenJev backend, benchmarked against the published zero-shot and supervised references; a fine-tuned readout served under the identical wire API, benchmarked against the zero-shot backend reported here; label-efficiency profiling of the fusion in the 50–200 labeled-example regime, which is the regime where the prior is most plausibly decisive against supervised baselines; temporal and external validation of the calibration claims; and a shadow deployment in which the confidence-gated reference policy runs silently alongside triage officers with override-reason logging. The framework, the harness, the Mac deployment recipe, and the ablation tooling for all of these steps are released with this preprint.
Data Availability Statement
simjev is available upon request.
Conflicts of Interest
The authors declare no competing interests.
References
- Razorback16. OpenJev: Open, Jev-compatible System-One decision server on DiffusionGemma. Apache License 2.0. Available online: https://github.com/razorback16/openjev.
- MedJev: Evaluating zero-shot System-One decision making in medicine. arXiv. 2026. Available online: https://github.com/Simonlee711/MedJev.
- TypeSafe. Jev documentation and System-One decision model guides. Available online: https://jev.guide.
- Defilippo, A.; Veltri, P.; Lió, P.; Guzzi, P.H. Leveraging graph neural networks for supporting automatic triage of patients. Sci. Rep. 2024, 14, 12548. [Google Scholar] [CrossRef] [PubMed]
- Defilippo, A.; Bertucci, G.; Zurzolo, C.; Veltri, P.; Guzzi, P. H. On the computational approaches for supporting triage systems. Interdiscip. Med. 2023, 1(3), e20230015. [Google Scholar] [CrossRef]
- Ahmed Aly, H. Patient Priority Classification. Kaggle dataset. Available online: https://www.kaggle.com/datasets/hossamahmedaly/patient-priority-classification.
- El Arab, R.E.; AlMoosa, O. The role of AI in emergency department triage: An integrative systematic review. Intensive Crit. Care Nurs. 2025, 104058. [Google Scholar] [CrossRef] [PubMed]
- Cabrera, D.; Thomas, J.; Wiswell, J.; et al. Accuracy of `My Gut Feeling’: Comparing System 1 to System 2 decision-making for acuity prediction, disposition and diagnosis in an academic emergency department. West. J. Emerg. Med. 2015, 16, 918–23. [Google Scholar] [CrossRef] [PubMed]
- Joynt, G.M.; Gopalan, D.P.; Argent, A.A.; et al. The Critical Care Society of Southern Africa consensus guideline on ICU triage and rationing (ConICTri). South Afr. Med. J. 2019, 109. [Google Scholar] [CrossRef] [PubMed]
- Downar, J.; Smith, M.J.; Godkin, D.; et al. A framework for critical care triage during a major surge in critical illness. Can. J. Anesth. 2022. [Google Scholar] [CrossRef] [PubMed]
- Johnson, A.E.W.; Pollard, T.J.; Shen, L.; et al. MIMIC-IV clinical database. PhysioNet 2023. [Google Scholar] [CrossRef]
- Fey, M.; Lenssen, J.E. Fast graph representation learning with PyTorch Geometric. ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019. [Google Scholar]
Figure 1.
Architecture of simjev. A triage record is serialized into a JSON state (Stage 1) and indexed against a historical cohort by a patient similarity index built on four similarity measures with thresholded edges (Stage 2). The index informs a frozen System-One decision model through two mechanisms: Mechanism A injects the outcome summary of the k most similar historical patients into the state, which the model answers with typed Choice/Score/Noul questions in a single forward pass; Mechanism B supplies a kNN prior probability to a log-odds fusion with recalibration (Stage 4). The decision policy is human-owned: confidence gates route uncertain cases to a triage officer, thresholds are institution-specific, and every probability vector is logged (Stage 5). Labeled outcomes (50–200 examples) flow back to recalibration and to the fusion weight w.
Figure 1.
Architecture of simjev. A triage record is serialized into a JSON state (Stage 1) and indexed against a historical cohort by a patient similarity index built on four similarity measures with thresholded edges (Stage 2). The index informs a frozen System-One decision model through two mechanisms: Mechanism A injects the outcome summary of the k most similar historical patients into the state, which the model answers with typed Choice/Score/Noul questions in a single forward pass; Mechanism B supplies a kNN prior probability to a log-odds fusion with recalibration (Stage 4). The decision policy is human-owned: confidence gates route uncertain cases to a triage officer, thresholds are institution-specific, and every probability vector is logged (Stage 5). Labeled outcomes (50–200 examples) flow back to recalibration and to the fusion weight w.

Figure 2.
Head-to-head evaluation on the iatriage cohort (outcome: red triage, prevalence 1.8%). Left: ROC curves for the kNN similarity prior, the GraphSAGE node-classification arm over the thresholded similarity network, and the fused model. Right: reliability curves (eight quantile bins) showing that log-odds fusion moves the GraphSAGE probabilities onto the diagonal: ECE decreases from 0.028 to 0.007 and Brier score from 0.024 to 0.010 at unchanged AUROC.
Figure 2.
Head-to-head evaluation on the iatriage cohort (outcome: red triage, prevalence 1.8%). Left: ROC curves for the kNN similarity prior, the GraphSAGE node-classification arm over the thresholded similarity network, and the fused model. Right: reliability curves (eight quantile bins) showing that log-odds fusion moves the GraphSAGE probabilities onto the diagonal: ECE decreases from 0.028 to 0.007 and Brier score from 0.024 to 0.010 at unchanged AUROC.

Table 1.
Held-out performance on the iatriage cohort (outcome: red triage, prevalence 1.8%). AUROC and AUPRC with 95% bootstrap confidence intervals (500 resamples); ECE with ten bins. The jev_fused_gnn arm fused the GraphSAGE probability with a heuristic-backend decision-model probability and is shown to document pipeline behavior; real-backend System-One results are reported separately in Table 2.
Table 1.
Held-out performance on the iatriage cohort (outcome: red triage, prevalence 1.8%). AUROC and AUPRC with 95% bootstrap confidence intervals (500 resamples); ECE with ten bins. The jev_fused_gnn arm fused the GraphSAGE probability with a heuristic-backend decision-model probability and is shown to document pipeline behavior; real-backend System-One results are reported separately in Table 2.
| Arm | AUROC [95% CI] | AUPRC | ECE |
|---|---|---|---|
| knn_only (similarity prior) | 0.932 [0.879, 0.976] | 0.366 | 0.004 |
| gnn_gcn | 0.975 [0.958, 0.990] | 0.547 | 0.063 |
| gnn_gatv2 | 0.976 [0.960, 0.990] | 0.564 | 0.037 |
| gnn_graphsage | 0.984 [0.972, 0.994] | 0.686 | 0.028 |
| jev_fused_gnn (GraphSAGE) | 0.984 [0.972, 0.994] | 0.686 | 0.007 |
Table 2.
Full-cohort evaluation against the real OpenJev backend under the solo-read protocol (6,552 encounters; 3,605 train / 982 tune / 1,965 test; outcome: red triage, prevalence 1.8%; 16 features). The two encoding columns differ only in how categorical fields are written into the JSON state (arbitrary integer codes vs. semantic strings); the kNN prior is encoding-independent and identical in both columns.
Table 2.
Full-cohort evaluation against the real OpenJev backend under the solo-read protocol (6,552 encounters; 3,605 train / 982 tune / 1,965 test; outcome: red triage, prevalence 1.8%; 16 features). The two encoding columns differ only in how categorical fields are written into the JSON state (arbitrary integer codes vs. semantic strings); the kNN prior is encoding-independent and identical in both columns.
| Integer codes | Semantic strings | |||
|---|---|---|---|---|
| Arm | AUROC [95% CI] | AUPRC | AUROC [95% CI] | AUPRC |
| knn_only (similarity prior) | 0.932 [0.879, 0.976] / 0.366 | — (same prior) | ||
| jev_zeroshot (plain state) | 0.397 [0.322, 0.485] | 0.014 | 0.474 [0.386, 0.549] | 0.017 |
| jev_augmented (Mechanism A) | 0.862 [0.797, 0.923] | 0.127 | 0.915 [0.856, 0.958] | 0.216 |
| jev_fused (Mechanism B) | 0.919 [0.849, 0.976] | 0.366 | 0.917 [0.850, 0.976] | 0.367 |
Table 3.
Configuration sensitivity of the zero-shot and augmented arms. Each row is a full evaluation configuration (probe rows: stratified diagnostic sample, Section 2.12). The zero-shot sign is unstable across state encodings and question compositions; the augmented arm is robust across all configurations. ECE for the panel zero-shot arm was 0.760 (probabilities saturating near 1.0); all solo-read arms are calibrated (ECE ).
Table 3.
Configuration sensitivity of the zero-shot and augmented arms. Each row is a full evaluation configuration (probe rows: stratified diagnostic sample, Section 2.12). The zero-shot sign is unstable across state encodings and question compositions; the augmented arm is robust across all configurations. ECE for the panel zero-shot arm was 0.760 (probabilities saturating near 1.0); all solo-read arms are calibrated (ECE ).
| Configuration | State | Questions | zeroshot AUROC [CI] | augmented AUROC [CI] |
|---|---|---|---|---|
| 14 features (loader defect) | integer codes | solo | 0.653 [0.567, 0.737] | 0.785 [0.705, 0.869] |
| 14 features (loader defect) | integer codes | 3-question panel | 0.336 [0.268, 0.404] | 0.919 [0.851, 0.976] |
| 16 features | integer codes | solo | 0.397 [0.322, 0.485] | 0.862 [0.797, 0.923] |
| 16 features | semantic strings | solo | 0.474 [0.386, 0.549] | 0.915 [0.856, 0.958] |
| probe, | codes, 14 fields | admission alone | 0.638 [0.525, 0.743] | — |
| probe, | codes, 14 fields | inside panel | 0.401 [0.288, 0.515] | — |
| probe, | codes, 16 fields | admission alone | 0.360 [—] | — |
| probe, | semantic, 16 fields | admission alone | 0.531 [—] | — |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.