Ensemble Voting of Large Language Models for Multiple-Choice Answer Prediction

Machine Reading Comprehension (MRC) has become a key benchmark for evaluating natural language processing systems, and multiple-choice question answering is widely adopted for assessing reasoning, comprehension, and commonsense inference capabilities. However, traditional embedding-based retrieval models often struggle with complex reasoning and lack sufficient external knowledge. Although Large Language Models (LLMs) have demonstrated strong language understanding and reasoning abilities, relying on a single model remains challenging due to hallucinations, model biases, and differences in training data, architectures, and domain expertise. As a result, no individual model can consistently achieve optimal performance across diverse multiple-choice question answering tasks. To address these limitations, we propose an ensemble framework for multiple-choice question answering. The framework incorporates a novel model selection mechanism that identifies model combinations with minimal error overlap, thereby maximizing error-correction capability. In addition, when disagreements occur among base models, a Thinking Model is dynamically invoked to analyze and resolve conflicting predictions, enabling advanced reasoning only when necessary. Experimental results show that the proposed framework consistently outperforms individual models across multiple datasets and previous ensemble approaches. At the same time, it significantly reduces inference time and computational cost compared to the strongest benchmark models.