Calib-Abstain: Uncertainty-Calibrated Self-Checking to Reduce Hallucinations in LLM Reasoning

Hallucinations in large language models (LLMs) remain a critical challenge for deploying these systems in high-stakes domains. We introduce Calib-Abstain, a framework that combines token-level uncertainty estimation with multi-path consistency checking to generate calibrated confidence scores. When confidence falls below a learned threshold, the system triggers tiered mitigation actions: retrieval augmentation, query reformulation, or abstention with explanation. Our approach employs temperature scaling and isotonic regression to minimize Expected Calibration Error (ECE). Experiments on mathematical reasoning (GSM8K, MATH), code generation (HumanEval), and general knowledge tasks (MMLU) demonstrate that Calib-Abstain reduces hallucination rates among answered queries by 24--31% while maintaining 80--88% coverage, with ECE improvements of 0.08--0.12 across datasets.