Fairness and minority exclusion have emerged as the central concerns of contemporary AI ethics. However, standard auditing practices often fail to capture harms affecting edge cases and marginalized groups. This article argues that this failure is structural: the act of "discretization"—converting continuous reality into discrete governance categories—inevitably produces a "residual." Drawing on German Idealism (Kant, Fichte, Schelling) and continental philosophy (Dilthey, Gadamer, Merleau-Ponty), we reconceptualize residuals not as mere noise but as "surprising facts" that should trigger abductive hypothesis revision. We critique current audit-by-checklist approaches as "rituals of verification" that obscure these residuals. This article makes three key contributions: (i) a structural diagnosis of residual production using systems theory and topology; (ii) a philosophical reconstruction of abductive revision as a hermeneutic necessity; and (iii) an institutional design proposal—specifically, the Residual Ledger and Category Revision Protocols—to operationalize "Open Schema" governance.