Silent Failure of Conformal Risk Certificates Under Imaging-Site Shift: A Cross-Site Study in Retinal Vessel Segmentation

Conformal risk control (CRC) can wrap a black-box segmentation network so that the expected false negative rate on a future case is provably at most a user-chosen level α, without distributional assumptions — except one: calibration and deployment data must be exchangeable. We show, for retinal vessel segmentation across three public fundus databases acquired with three cameras on three populations (DRIVE, STARE, CHASE-DB1), that this assumption fails in the way that matters most for safety. A certificate calibrated on one site and transferred to a harder site misses 27% of vessel pixels while still reporting α = 0.15 — a silent, unsafe violation invisible without target labels. We prove that monotone probability recalibration (temperature, Platt, isotonic) leaves the CRC mask and its risk exactly unchanged, and show empirically that importance-weighted CRC, though valid in theory, degenerates at realistic cohort sizes: a twosample discriminator on encoder features separates clinical sites almost perfectly (AUC 1.00), so estimated density ratios are effectively unbounded and no informative threshold survives. We therefore recast the problem as detection and deferral. An unlabelled, image-only diagnostic reliably flags out-of-support sites, our SHARC wrapper controls risk on the sub-population it accepts, and as few as six labelled target images restore a valid, informative certificate at both shifted sites. For regulated deployment, a transported risk bound should be treated as unverified until the site is shown to lie within calibration support or is re-certified with a small labelled sample.