TriageRAG: Confidence-Based Triage for Patent Examination Outcome Prediction

Predicting patent examination outcomes under 35 U.S.C. §103 is inherently difficult because obviousness determinations require context-sensitive legal reasoning over prior art combinations that cannot be captured by surface-level text patterns alone. Existing automated approaches optimize for aggregate accuracy but offer no principled criterion for when their predictions should be trusted and when practitioner review remains necessary. We present TriageRAG (T-RAG), a two-stage decision-support framework that addresses this gap by treating classifier confidence as an explicit routing signal. A fine-tuned ModernBERT-large model first produces a prediction together with a calibrated confidence score; high-confidence predictions are delivered directly, while uncertain cases are escalated to a Large Language Model (LLM) that reasons over balanced retrieval from a knowledge base of 50,000 granted patents and 50,000 §103-rejected applications with full examiner Office Action text. This balanced retrieval ensures that escalated predictions are grounded in auditable, bidirectional evidence rather than opaque model parameters. Empirical evaluation on USPTO patent applications confirms that the confidence threshold provides a reliable escalation criterion: LLM verification yields the largest accuracy gains precisely on the cases the classifier is least certain about, and confidence-based routing is statistically superior to random routing at equivalent LLM utilization rates. Ablation studies further characterize the accuracy–cost trade-off across threshold values and reveal domain-specific reliability profiles that practitioners can use to calibrate their trust in system outputs by technology area. T-RAG thus serves as a transparent decision-support tool that not only predicts examination outcomes but provides structured guidance on where additional scrutiny is warranted.