Explainable Multi-Hop Question Answering: A Rationale-Based Approach

1. Introduction

Recent advancements in artificial intelligence (AI) have led to significant improvements across various domains. In particular, large language models (LLMs) [1,2,3,4] have transformed the field of natural language processing (NLP). By learning from vast amounts of data, LLMs understand linguistic context, meaning, and structure, enabling accurate generation and comprehension of natural language. Despite their remarkable capabilities, LLMs encounter critical challenges related to the opacity and reliability of their reasoning processes [5,6,7]. For instance, even when users prompt LLMs to provide a rationale for a response, they often remain unsure how that rationale is generated [8]. This lack of transparency raises concerns about why the model produces a response and what rationale underlies its decisions. As a result, users may struggle to trust the model’s responses, leading to potential risks regarding the accuracy and reliability of the information provided. This lack of reliability can reduce users’ trust in the model, potentially leading them to seek alternative sources of information. To address these concerns, research in explainable artificial intelligence (XAI) aims to enhance the transparency of AI systems by making their reasoning processes more understandable, thereby fostering greater trust in their outputs [9,10,11]. Therefore, XAI must provide clear explanations of how AI systems reach their responses, offering intuitive insights that facilitate user trust and understanding.

In this paper, we define the criteria for XAI models as follows: (1) The rationales provided by the model must be reliable and understandable to users. (2) Users should be able to comprehend the model’s operation through its reasoning process. (3) The model should be capable of providing rationales without relying on annotations attached to the data. (4) The rationales provided by the model should align with its predictive outputs. To meet these criteria, we propose a framework that satisfies these requirements and apply it to the multi-hop question answering task, a representative task related to XAI.

Multi-hop question-answering tasks involve identifying multiple supporting sentences from a given set of documents, which serve as evidence for inferring an answer. These supporting sentences serve as rationale for deriving accurate responses. Since this task is a complex process that requires understanding and utilizing the relationships between sentences across multiple documents, it is crucial to identify reliable rationales. Most multi-hop question answering research [11,12] consists of two main components: a supporting sentence identification module and a reading comprehension module. The supporting sentence identification module identifies the sentences necessary for inferring the answer. It identifies supporting sentences within the documents based on retrieval models or supervised learning [13,14]. The reader then uses the sentences selected by the supporting sentence identification module to infer responses to the given questions. However, these approaches have several limitations. Firstly, the supporting sentence identification module requires explicit annotations for rationale. Even with these annotations, clearly explaining how the rationale is derived remains challenging. Secondly, the reader module often considers the entire input sequence composed of the retrieved segment as the rationale. This makes it difficult to clearly explain the reader model’s reasoning process, which may not meet the requirements of XAI. Consequently, traditional multi-hop question-answering approaches are not well-suited to open-domain environments where annotated rationales are difficult to obtain or where model transparency is crucial.

To address these issues, this paper proposes a framework for explainable multi-hop question answering that makes the model’s reasoning process explicit and enables rationale inference without annotations. The proposed model is designed based on a pointer network and is trained using a training strategy that guides it to infer rationale sentences. Additionally, the model ensures factuality by extracting rationales from documents and presenting them at the sentence level to enhance user understanding. Furthermore, the process of selecting rationale sentences using the pointer network provides insights into the model’s reasoning process, and demonstrates strong performance without the need for explicit annotations. This approach meets the requirements of XAI and achieves strong rationale inference performance in experiments.

2. Related Works

XAI aims to explain a model’s decision in ways that users can understand [15,16,17]. Most XAI research [18,19,20,21] focuses on interpreting the model’s inference process or identifying key words, phrases, or sentences that influence predictions, rather than relying directly on annotated rationale labels. For example, [22] proposed aligning textual elements in natural language inference to analyze sentence correspondence, while using masking techniques to provide token- or phrase-level rationale. Similarly, [10] used trainable masks to block or activate specific nodes or edges in a graph, thereby removing irrelevant information and extracting key rationales. However, these studies have limited scope and applicability.

Multi-hop Question Answering requires combining multiple sentences from a given document set to answer complex questions. This task requires the clear identification of supporting sentences necessary for deriving the answer. [23] attempted to explain the rationale by classifying whether each sentence qualifies as supporting evidence. However, their approach does not sufficiently address interactions between sentences and has limitations in explaining complex, multi-document questions. [13] demonstrated the potential of utilizing annotation-based rationales through supervised learning. However, annotation-dependent methods face challenges in open-domain environments. To address this, [24] and [25] introduced complex datasets without annotations and emphasized the need for effective unsupervised learning methods. [26] and [23] proposed methods for finding rationale at the sentence level but mainly focused on dependencies between two pieces of information. In contrast, models like HUG [11] sought to address limitations by performing multi-hop inference through document and sentence predictions, though the explainability of these models’ operations remains insufficiently explored. Existing XAI approaches still face challenges in explainability and transparency. For instance, RAG [14] treats evidence as latent variables but overlooks inter-document connections. [9] successfully generated pseudo-evidence annotations through semi-supervised learning for specific QA tasks, but this approach is limited to certain types of multi-hop questions. To overcome these limitations, ongoing research aims to develop explainable multi-hop QA methods [11,27,28]. However, a general approach [29,30] that clearly explains supporting sentences and is applicable across various domains is still needed. Therefore, this paper proposes a rationale reasoning framework that effectively infers the rationale without annotations in open-domain environments and provides explanations of the model’s reasoning process. The proposed model is applicable to various tasks without structural constraints and has demonstrated superior performance on multi-hop question answering, relation extraction, and sentence classification tasks.

3. Methodology

Multi-hop question answering identifies the supporting sentences from a document set $D=\{d_0,d_1,...,d_{\left|D\right|-1}\}$ and infers the answer to question Q. Here, $\left|D\right|$ refers to the size of the document set. Figure 1 illustrates the structure of the proposed model. The proposed model comprises a pre-trained language model [31] and a pointer network [32]. The pre-trained language model encodes the documents, while the pointer network selects the information necessary for answer inference. The final hidden states of the pointer network, which accumulate rationale information, are used for the final answer.

Input Encoder Specifically, the input sequence is defined as follows:

$$T=\{\{t_0^0,...,t_{L-1}^0\},...,\{t_0^{\left|D\right|-1},...,t_{L-1}^{\left|D\right|-1}\}\}\in {\mathbb{R}}^{\left|D\right|\times L}$$

(1)

where T consists of a [CLS] token, a tokenized question, an [SEP] token, a tokenized document, and a final [SEP] token. Here, L refers to the maximum number of tokens in the input sequence. The sequence T is then processed by an encoder to obtain a token vector sequence V. Subsequently, sentence vectors $S=\{s_0^0,s_1^0,...,s_{\left|d\right|-1}^{\left|D\right|-1}\}\in {\mathbb{R}}^{\left|S\right|\times h}$ are generated using the encoded token vector sequence V using mean pooling. Here, $\left|d\right|$ refers to the maximum number of sentences constituting each document, $\left|S\right|$ refers to the total number of sentences in the document set D, and h denotes the output vector size of the pre-trained model.

Pointer Network The pointer network selects sentences through attention mechanisms with S and is guided to choose supporting sentences using the proposed training strategy. The initial state of the pointer network is set to $None$, and the initial input, $inpu{t}_0$, is as shown in Equation 2 below:

$$\begin{array}{c}\hfill inpu{t}_0={\displaystyle \frac{1}{\left|D\right|}}\sum _{d=0}^{\left|D\right|-1}{s}_0^d\end{array}$$

(2)

The output of the pointer network, $h_n$, is as shown in Equation 3.

$$\begin{array}{c}\hfill h_n=GRU(inpu{t}_n,h_{n-1})\end{array}$$

(3)

Here, n refers to the number of decoding steps, and $inpu{t}_n$ and $h_{n-1}$ represent the input to the RNN [33] and the previous RNN output, respectively. The context vector $C_n$ is generated by incorporating important sentence information through an attention operation between the output $h_n$ produced by the RNN and the sentence representations S, as shown in Equation 4 below:

$$\begin{array}{cc}\hfill & score(S,h_n)=h_n{W}_a{S}^T\hfill \\ \hfill & {\alpha }_n=softmax\left(score(S,h_n)\right)\hfill \\ \hfill & C_n=\sum _k^{\left|D\right|}{\alpha }_{n,k}{S}_k\hfill \end{array}$$

(4)

In this equation, $W_a$ represents a trainable weight matrix, and $score(S,h_n)$ denotes the importance of each sentence, calculated as the inner product between the sentence representations S and the RNN output $h_n$ at the current decoding step. ${\alpha }_n$ refers to the $score(S,h_n)$ normalized via the softmax function, and $C_n$ represents the weighted sum vector of S with respect to ${\alpha }_n$. The context vector $C_n$ is concatenated with the RNN output $h_n$ to form the next input vector $inpu{t}_{n+1}$, as described in Equation 5 below:

$$inpu{t}_{n+1}=W_{in}(C_n\oplus h_n)$$

(5)

In the equation above, $W_{in}$ represents a trainable weight matrix. As shown in Figure 1, we consider the final input, $inpu{t}_N$, accumulated through the pointer network, as the rationale vector. This is used to generate the probability distribution (${\widehat{y}}_{start}$, ${\widehat{y}}_{end}$) for the answer positions, and the corresponding equation is given by Equation 6.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {\widehat{y}}_{start}=V{W}_s{\left(inpu{t}_N\right)}^T\in {\mathbb{R}}^{\left|D\right|\times L}\hfill \\ \hfill & {\widehat{y}}_{end}=V{W}_e{\left(inpu{t}_N\right)}^T\in {\mathbb{R}}^{\left|D\right|\times L}\hfill \end{array}\end{array}$$

(6)

In the above equation, the probability distributions ${\widehat{y}}_{start}$ and ${\widehat{y}}_{end}$ are obtained by multiplying the token representations V with the trainable weight matrices $W_s$ and $W_e$, respectively, followed by multiplication with the rationale vector $inpu{t}_N$.

Training In this paper, we train the proposed model using two loss functions: one for answer inference and another for guiding the pointer network in selecting the supporting sentences. First, the model is trained by minimizing the cross-entropy between the predicted probability distributions, ${\widehat{y}}_{start}$ and ${\widehat{y}}_{end}$, and the ground-truth answer positions $y_{start}$ and $y_{end}$, defined as follows:

$$\begin{array}{cc}\hfill & L_s=-\sum y_{start}log\left({\widehat{y}}_{start}\right)\hfill \\ \hfill & L_e=-\sum y_{end}log\left({\widehat{y}}_{end}\right)\hfill \\ \hfill & L_{span}=(L_s+L_e)/2\hfill \end{array}$$

(7)

The proposed framework is trained to minimize the negative log-likelihood (NLL) described above, ensuring effective task performance. Specifically, during the training, the model adjusts its predicted label distribution to closely align with the correct label distribution by incorporating the information accumulated through the sentence selections of the pointer network. This mechanism enhances the model’s ability to generate reliable and well-informed predictions. Furthermore, to guide the pointer network in accurately selecting rationale sentences, an auxiliary loss function is introduced. This auxiliary loss function plays a crucial role in refining the sentence selection process, encouraging the model to make more precise decisions when identifying important rationale sentences. As a result, the selected sentences contribute meaningfully to the model’s final predictions, ultimately improving both interpretability and reliability.

The corresponding equation is presented in Equation 8 below:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill L_D\left(r\right)=\sum _{n=0}^{N-1}[{\alpha }_d\sum _{s\in r}log\left(P\left(s\right|e_n)\right)+(1-{\alpha }_d)\sum _{s\in S-r}log\left(P\left(s\right|e_n)\right)]\\ \hfill L_G\left(r\right)=\sum _{n=0}^{N-1}[{\alpha }_g\sum _{s\in r}log\left(P\left(s\right|e_n)\right)+(1-{\alpha }_g)\sum _{s\in S-r}log\left(P\left(s\right|e_n)\right)]\end{array}\end{array}$$

(8)

$$\begin{array}{c}\begin{array}{c}\hfill {\alpha }_d=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\mathit{F}}_1(y_r,y_D)>Threshold\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\\ \hfill {\alpha }_g=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\mathit{F}}_1(y_r,y_G)>Threshold\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}\end{array}$$

(9)

In this equation, $P\left(s\right|e_n)$ represents the probability that the pointer network selects sentence s at the n-th decoding step. $y_D$ is the predicted answer based on the entire input documents D, while $y_r$ is the predicted answer from the reasoning path r selected by the pointer network. In other words, if an inference similar to the full context can be achieved with only the sentences selected by the pointer network, the probability $P\left(s\right|e_n)$ of the selected sentences is increased; otherwise, probabilities of unselected sentences increase. Similarly, $L_G$ is calculated by comparing the annotated ground truth $y_G$ with $y_r$. The final loss function $L_{total}$ is defined as shown in Equation 10.

$$L_{total}=L_{span}+\lambda ·(L_D\left(r\right)+L_G\left(r\right))$$

(10)

In the above formula, $\lambda$ represents a weighting coefficient.

Beam Search Decoding This section explains the process of extending the pointer network, as shown in Figure 2. Through beam search decoding [34], the model generates K reasoning paths $R=\{r^1,r^2,...,r^K\}$, evaluates each path, selects the optimal reasoning path $r^{k*}$ for answer inference, and uses it during training. Each reasoning path $r^k$ is evaluated by comparing the predicted answer $y_D$ based on the entire document, the predicted answer $y_{r^k}$ based on $r^k$, and the ground-truth answer $y_G$, using the sum of ${\mathit{F}}_1$ scores. This process is illustrated in Figure 3.

We select the best path, $r^{k*}$, which maximizes the ${\mathit{F}}_1$ score among the reasoning paths and use it to train the model using the following equation.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & f\left(r^k\right)={\mathit{F}}_1(y_{r^k},y_D)+{\mathit{F}}_1(y_{r^k},y_G)\hfill \\ \hfill & k^{*}=arg\underset{K}{max}f\left(r^k\right)\hfill \\ \hfill & L=L_{span}+\lambda ·(L_D\left(r^{k^{*}}\right)+L_G\left(r^{k^{*}}\right)\hfill \end{array}\end{array}$$

(11)

4. Experiments

4.1. Dataset and Experimental Setup

The primary task addressed in this study is multi-hop question answering. To assess the effectiveness of the proposed approach, we conducted experiments using two multi-hop question answering datasets [12,35]. Additionally, to emphasize that the proposed model is task-agnostic and capable of inferring rationales for its predictions across various tasks, we conducted additional experiments on a document-level relation extraction dataset [36] and a binary sentiment analysis dataset [37]. Through these experiments, we aimed to demonstrate the model’s consistent performance across diverse tasks and confirm that its rationale extraction capability is not limited to a specific task.

The proposed model utilizes the pre-trained ELECTRA-base [31] as its encoder. The weighting coefficient $\lambda$ in Equations 10, and 11 was set to 0.1. Furthermore, the pointer network was constrained to extracting two or three sentences, depending on the dataset used in the experiments.

HotpotQA This dataset [12] is a large-scale multi-hop question answering dataset designed to integrate information across documents to answer questions. It contains approximately 112,000 questions and corresponding answers covering a wide range of topics. For each question, 10 related documents are provided, containing supporting sentences that the model must reference to infer the correct answer. In our experiments, we used 90,564 samples for training and 7,405 samples for development under the distractor setting.

MuSiQue This dataset [35] is designed to train and evaluate question answering tasks requiring multi-hop reasoning over text, similar to HotpotQA. It comprises approximately 25,000 questions with corresponding answers, where each question consists of several sub-questions. This structure enables the evaluation of the model’s ability to integrate information from various sources to derive answers. The proposed model is trained to infer answers to the initially provided questions without directly addressing the sub-questions. The dataset consists of 19,144 samples and 2,417 development samples.

DocRed This dataset [36] is a large-scale resource for document-level relation extraction, focusing on identifying relationships between entities across entire documents rather than within individual sentences. It contains 132,375 entities and 56,354 relationships extracted from 5,053 Wikipedia documents, encompassing 96 distinct relation types. Notably, each relation instance is accompanied by supporting sentences for relation extraction. The objective task includes identifying pairs of entities that have existing relations among all possible combinations of entities. However, in our experiments, relation extraction was performed only on entity pairs with existing relations, and documents with fewer than three sentences were excluded. The final dataset used for training comprises 56,195 samples, while the evaluation dataset includes 17,803 samples.

IMDB This dataset [37] is designed for sentiment analysis, aiming to predict the sentiment (positive/negative) by analyzing movie review texts. It includes a total of 50,000 movie reviews, with each review labeled as either positive or negative. The dataset is split into 25,000 training samples and 25,000 testing samples, with each set containing an equal number of positive and negative reviews.

Detailed statistics of the datasets used in the experiments are presented in Table 1.

As shown in Table 1, we used four publicly available datasets for our experiments, utilizing the train and test sets for quantitative evaluation of both answer prediction and rationale inference. However, since the IMDB dataset does not include rationale annotations, it cannot be directly used for quantitative evaluation of rationale inference. Therefore, we sampled a subset of the test set (Sampled Test set) and evaluated rationale inference using a GPT-based model.

4.2. Baseline

The proposed framework is designed to be adaptable across various tasks. To assess its generalizability, we adjusted the output layer according to each task. In this process, unlike in the multi-hop question answering task, the rationale vector $inpu{t}_N$ is concatenated with the $\left[CLS\right]$ vector to generate the final probability distribution $\widehat{y}$. The detailed structure of this process is illustrated in Figure 4 below.

At this stage, for the classification task, the best reasoning path $r^{k*}$ is determined using KL Divergence Loss [38] instead of the ${\mathit{F}}_1$ score. Specifically, the selected sentence subset $r^{k*}$ is the one that yields a prediction most similar to the prediction obtained by using the entire input sequence. Detailed implementation for each task is provided in Appendix A.

4.3. Metrics

As a performance metric, we set the ${\mathit{F}}_1$ score for the model’s predictions. For HotpotQA, MuSiQue and DocRED, which contain annotated rationale, we compute not only the ${\mathit{F}}_1$ score of the answers but also the precision, recall, and ${\mathit{F}}_1$ score of the rationale. The ${\mathit{F}}_1$ score of the rationale is calculated at the sentence level. Furthermore, we define a model as having high rationale inference capability if it provides appropriate supporting rationale, even when its predictions are incorrect. Therefore, we determined that comparing the model’s rationale inference capabilities based only on annotated rationales is insufficient, so we utilize GPT-4o mini for this evaluation. Specifically, we instruct GPT-4o mini to evaluate the supporting rationale for predictions by providing the task description, the model’s predicted answers, and the inferred rationale sentences. We conduct the evaluation on a scale from 0 to 2, depending on how well the rationale sentences support the predictions. The evaluation criteria are set as follows:

• 0 points: The provided rationale does not substantiate or is irrelevant to the model’s prediction at all.
• 1 point: The provided rationale somewhat supports the model’s prediction but is not decisive; it is partially inferred but unclear.
• 2 points: The provided rationale fully substantiates the model’s prediction, and the same answer can be inferred solely based on the given rationale.

The GPT-4o mini evaluation was performed on a dataset (Sampled Dataset) consisting of 1,000 randomly selected samples from each evaluation dataset. The evaluation scores were normalized to a range of 0 to 1. The prompts used in the experiments are detailed in Appendix B.

4.3.1. Prompt Details for GPT-Based Evaluation

In this section, we provide a detailed explanation of the prompts used to guide GPT-4o mini in evaluating the model’s rationale inference capabilities. The prompts were structured to present GPT-4o mini with three key pieces of information: the task description, the model’s predicted answer, and the rationale sentences inferred by the model. The goal of the evaluation was to measure how effectively the inferred rationale sentences supported the model’s predictions. Each sample provided GPT-4o mini with the model’s predicted answer and the corresponding rationale, and the model was instructed to score the relevance of the rationale on a scale from 0 to 2, according to predefined criteria. The following elements were included in each prompt:

• Task Description: A clear explanation of the evaluation task.
• Predicted Answer: The model’s predicted answer for the given question.
• Rationale Sentences: The evidence sentences inferred by the model, to be evaluated by GPT-4o mini.

These elements formed the core of each evaluation prompt.

4.4. Comparison Models

To evaluate the rationale inference performance of our proposed model against existing approaches, we selected the following comparison models. Models trained on annotated evidence from the dataset were excluded from this evaluation.

HotpotQA and MuSiQue: BM25 [39] and RAG [14] serve as baselines retrieval models that were not trained on the experimental dataset. RAG was modified to search at the sentence level to match the characteristics of the dataset, providing three fixed sentences for performance evaluation. The semi-supervised approach [9] involves constructing silver rationale annotations in advance and learning through an RNN decoder to infer the rationale. HUG [11] is a state-of-the-art unsupervised model for multi-hop question answering. It identifies the optimal rationale by combining sentences from all documents and predicts the answer based on this rationale. [40] applies a multi-stage retrieval method at both the document and sentence levels, based on BM25. Their approach demonstrated superior performance compared to existing retrieval methods and showed strong performance in answering complex questions such as those in multi-hop question answering. The upper bound represents the performance of models trained on annotated rationales, similar to the structure of the proposed model. It provides an upper bound on the rationale inference performance that the proposed model can potentially achieve.

DocRED and IMDB: GPT_pred illustrates the performance of GPT-4o mini [41] in a zero-shot setting. The prompts used in the experiments are detailed in Table A5 and Table A6. For DocRED, GPT_pred refers to the performance GPT-4o mini when instructed to perform both relation extraction and evidence inference simultaneously. For the IMDB dataset, GPT_pred is prompted to perform sentiment analysis while simultaneously identifying key sentences that influence the classification.

This evaluation assesses the model’s ability to infer rationales in an unsupervised manner without access to annotated evidence, demonstrating its effectiveness in extracting supporting rationales directly from the text. In this experiment, GPT is given ground-truth labels and supporting sentences from the dataset, independent of GPT_pred’s inference results, to assess how effectively the provided rationale supports the correct answers.

4.5. Experimental Results

4.5.1. Quantitative Evaluation

Table 2 presents the results for HotpotQA. The proposed model demonstrates the highest performance in rationale inference compared to unsupervised learning-based models. Specifically, compared to the semi-supervised model, the proposed model achieves superior performance, providing a significant advantage over methods that rely on explicit silver-label learning. Compared to the approach by [40], which searches for five fixed sentences, the proposed model exhibits higher recall performance in rationale sentence inference. Since the proposed model extracts only three sentences, the performance gap is expected to be even larger when extracting the same number of sentences. The proposed model and HUG achieve comparable performance. However, HUG, which explores the optimal reasoning path by considering all combinations of the given documents and sentence pairs, has high time complexity. In contrast, the proposed model, which adds only a short decoding process to the reading model, offers similar performance to HUG while providing a speed advantage. Furthermore, considering that recall is a more important performance metric than precision when providing rationales to users, the proposed model demonstrates approximately 95% of the recall performance of the upper bound, even without annotated evidence.

Table 3 presents the results for MuSiQue. Similar to HotpotQA, the proposed model demonstrates the highest performance compared to unsupervised rationale inference models.

Table 4 presents the experimental results for DocRED. The proposed model achieves an Answer ${\mathit{F}}_1$ score of 81.8, significantly outperforming GPT_pred, which score of 41.8. This demonstrates the superior performance of the proposed model in relation extraction compared to GPT-based models. Additionally, the rationale ${\mathit{F}}_1$ scores for GPT_pred is 21.8, which is considerably lower than the 54.9 achieved by our model. This result indicates that the proposed model excels in rationale sentence extraction relative to GPT models. However, since these experiments are based on rationale sentences attached to the data, they provide limited insight into the comparative rationale inference capabilities of each model. To address this, the next section will further analyze how well the inferred answers and rationale sentences correspond to each other.

4.5.2. GPT Score Evaluation

To evaluate how effectively the rationale sentences inferred by the model support the final predictions, we conducted an experiment using the GPT-4o mini model. This experiment measures the degree to which the inferred rationale sentences convincingly justify the predicted answers based on both model-generated rationales. Additionally, we included GPT_gold, which utilizes ground truth answers and rationale sentences from the dataset, and GPT_pred, which performs the dataset task while simultaneously inferring rationale sentences, as comparison models. Table 5 presents the evaluation results, using the GPT-4o mini model, showing how well the rationale sentences inferred by the model support the predicted answers across four datasets (HotpotQA, MuSiQue, DocRED, and IMDB). We considered GPT_pred, which performs the dataset task while simultaneously inferring rationale sentences, and GPT_gold, which uses the ground truth answers and rationale sentences attached to the dataset, as comparison models.

First, regarding GPT_gold, we observe that its score does not reach 100%, even when it has access to the correct answers and rationales. This suggests a misalignment between the evaluation criteria of the datasets and those used by the GPT-4o mini model. Next, GPT_pred achieves over 90% of GPT_gold’s performance across most datasets. This demonstrates that GPT-based models can achieve relatively high rationale inference without strictly adhering to the rationale sentences provided in the ground truth data. In particular, for the HotpotQA and MuSiQue datasets, the performance gap between GPT_pred and GPT_gold is minimal, indicating that GPT models can infer evidence sentences that closely approximate the quality of the actual data. However, for the DocRED dataset, there is a significant performance gap between GPT_pred and GPT_gold (51.0 vs. 65.0). Considering the inherent limitations of GPT in relation extraction tasks, this result reflects the model’s constraints in inferring accurate rationale sentences. Although GPT_gold results are unavailable for the IMDB dataset, GPT_pred achieves a high score of 95.4. This suggests that GPT models are capable of inferring highly relevant rationales in tasks involving single-document analysis, performing better in simpler structures compared to those requiring complex document linking. Overall, while GPT-based models exhibit strong performance across various datasets, there is a notable decrease in rational inference accuracy for tasks with lower performance, and a greater divergence from the annotated rationales is observed in these cases. The proposed model also demonstrates generally strong rationale inference performance, though a significant decrease is observed in the MuSiQue experiments. However, compared to the answer inference performance in earlier experiments, the rationale inference performance remains higher. This suggests that the proposed model can infer suitable rationale even when the predicted answer is incorrect. Finally, the overall scores for GPT-4o mini and the proposed model are 79.9 and 79.8, respectively, indicating very similar performance. Nevertheless, despite having only approximately 1/80 the parameters of GPT-4o mini, the proposed model demonstrates comparable rationale inference capabilities. Additionally, the proposed model can be considered an effective rationale inference framework suitable for XAI, as it demonstrates the model’s reasoning process through the pointer network indirectly guided to select rationale sentences. The detailed prompts for the experiment are provided in Appendix C.

4.5.3. Ablation Study

To analyze the impact of individual components on the performance of the proposed model, we conducted an ablation study by selectively removing or modifying key elements. This analysis allows us to assess the significance of each component in the learning and inference processes. Table 6 presents the results of the ablation experiments conducted on the proposed model. It is evident that removing the loss function significantly reduces the effectiveness of learning in the rationale sentence extraction process. This suggests that without an appropriate loss function tailored for the pointer network, the model’s ability to accurately infer rationale sentences is compromised. Additionally, the results show that beam search decoding allows for more accurate evidence inference compared to greedy decoding, enabling more refined selections. Overall, the proposed model demonstrates high performance in rationale sentence extraction by effectively utilizing both the loss function and beam search decoding, with particularly strong results in terms of recall. This leads to the conclusion that applying an appropriate learning loss and employing beam search for decoding play crucial roles in rationale sentence extraction using a pointer network.

Given the importance of the loss function in guiding the pointer network, we further investigate how the threshold values used in the loss function affect rationale inference performance. Specifically, we examine the impact of the threshold in Equation 9 on the model’s ability to extract rationale sentences accurately. Figure 5 illustrates the changes in rationale inference performance based on the threshold values used to determine ${\alpha }_d$ and ${\alpha }_g$ in the loss function (Equation 8). According to the graph, the model achieves high performance when the threshold value is between 0.4 and 0.6, with the highest rationale inference performance occurring at 0.5. Moreover, the performance sharply declines as the threshold approaches either 0 or 1. This indicates that overly strict or lenient criteria for evaluating rationale sentences can hinder the learning of the pointer network. Thus, selecting an appropriate threshold value is crucial for improving the performance of the pointer network.

4.5.4. Further Analysis

To evaluate whether the model maintains consistent predictions when provided with only the selected rationale sentences instead of the entire document, we conducted an additional experiment. This analysis investigates the degree to which the selected rationale sentences contribute to the model’s reasoning process. Table 7 presents the results verifying whether the model can infer identical predictions using only the selected rationale sentences, compared to when the entire document is provided, as mentioned in the introduction. The experiment was conducted on the HotpotQA evaluation dataset. In the table, “Proposed Model (All Document ⇔ Only Rationale Sentences Input)" shows the ${\mathit{F}}_1$ score comparing the predictions inferred by the proposed model using the entire document versus those inferred using only the selected rationale sentences. Additionally, for comparison, we measure the ${\mathit{F}}_1$ scores between the inferred predictions and the dataset’s ground truth in both cases: “Ground Truth ⇔ All Document Input" and “Ground Truth ⇔ Only Rationale Sentences Input". Here, “Test set ${\mathit{F}}_1$" represents the evaluation results on the entire HotpotQA test set, while “Sampled Test set ${\mathit{F}}_1$" measures the performance only on the subset of the test set where the predictions inferred from the entire document exactly match the ground truth.

The evaluation results in Table 7 demonstrate the consistency of predictions made by the proposed model when using the entire document compared to when using only the selected rationale sentences. The key findings are as follows: In the results for “Proposed Model (All Document Input ⇔ Only Rationale Sentences Input)", the Test set ${\mathit{F}}_1$ score is 84.9, showing a high level of consistency between the answers predicted based on the entire document and those predicted using only the rationale sentences. Furthermore, the Sampled Test set ${\mathit{F}}_1$ score increases to 93.3, indicating that the selected rationale sentences effectively preserve essential information for generating accurate predictions. This score is particularly noteworthy in cases where the predictions based on the entire document exactly match the Ground Truth. As a result, the proposed model demonstrates strong consistency in predictions even when only the selected rationale sentences are provided, achieving results comparable to when the entire document is used. Additionally, the increase in the Sampled Test set ${\mathit{F}}_1$ score highlights the robustness of rationale-based predictions in scenarios where predictions align with the Ground Truth.

5. Conclusion

We propose a framework for effective rationale inference in explainable multi-hop question answering. Our model utilizes a pointer network to accumulate important information and infer answers, effectively selecting evidence sentences even in the absence of annotations. Furthermore, the model’s reasoning process is interpretable through the pointer network’s sentence selection. Compared to prior multi-hop question-answer models, our approach achieves state-of-the-art performance across multiple metrics and provides supporting evidence even when predictions are incorrect. Moreover, compared to the recently released GPT-4o mini, our model-despite having only approximately 1/80 of the parameters-exhibited comparable rationale inference capabilities.

6. Patents

This section is not mandatory, but may be added if there are patents resulting from the work reported in this manuscript.