Moving Explanations Closer to Humans: A Systematic Review of Comparisons of Explainable AI Implementations

1. Introduction

The field of Artificial Intelligence (AI) is rapidly expanding, with AI technologies being increasingly integrated into everyday life. With the current, often opaque, AI systems, there are concerns about the lack of transparency and interpretability. The General Data Protection Regulation (GDPR) has emphasized the importance of the explainability of AI systems and that end-users have the “right to explanation” (Díaz-Rodríguez et al., 2023; Goodman & Flaxman, 2017). As a solution to these concerns, there is a growing interest in Explainable AI (XAI), which aims to increase the transparency of AI systems and make the outcomes understandable to users, by providing explanations of the inner workings of the algorithm (Samek & Müller, 2019).

In the early stages of the field, XAI research primarily focused on providing explanations for experts and AI developers, often with the goal of improving model accuracy and system performance (Vieira & Digiampietri, 2022). Consequently, many XAI solutions were based on developers’ intuitions of a ‘good’ explanation, rather than on knowledge from social sciences or empirical evidence (Miller, 2019). More recently, research has shifted towards more participatory approaches to XAI, actively involving end-users in the design and evaluation of explanations (Cabitza et al., 2023; Laato et al., 2022; D. Wang et al., 2019). As AI systems are increasingly deployed in non-technical, yet safety-critical domains, such as healthcare (e.g., Bussone et al., 2015; Cai et al., 2019), criminal justice (e.g., Dodge et al., 2019), and education (e.g., Farrow, 2023), the primary users are often lay users without technical expertise (Islam et al., 2022). For these users, explanations play a crucial role in enabling them to interpret, evaluate, and appropriately weigh AI-generated recommendations when making decisions. Identifying explanation approaches that are broadly useful and meaningful for end-users is therefore important for effective human–AI interaction (Amershi et al., 2014; Samek & Müller, 2019).

However, it remains unclear which factors contribute to effective explanations. Existing user studies report mixed findings regarding the impact of XAI on human-AI interaction (Haque et al., 2023; Rong et al., 2024; Schemmer et al., 2022). While many studies indicate that explanations can improve outcomes such as trust, understanding, usability, and collaboration performance (Haque et al., 2023; Rong et al., 2024), others find limited or no significant benefits of XAI (Schemmer et al., 2022). A potential reason for these inconsistent results is the substantial heterogeneity across studies, including differences in XAI implementations, explanation designs, application domains, and end-user evaluation methods. This diversity makes it difficult to identify which characteristics of XAI implementations, or which contextual factors within specific domains, contribute to effective human-AI interaction, as well as to determine whether observed effects generalize across use cases and domains. Addressing this challenge requires systematic comparisons of the factors that influence key outcome measures.

Numerous frameworks have been proposed to suggest factors that contribute to effective explanations. In their scoping review, Cortiñas-Lorenzo et al. (2025) identified as many as 73 frameworks that address different aspects of XAI design, implementation, and evaluation. Despite differences in terminology, scope, and emphasis, these frameworks show quite some overlapping factors that are considered important for a 'good' explanation. Some frameworks primarily focus on guiding the design of explanations (e.g., Adhikari et al., 2022; Cabour et al., 2023; D. Wang et al., 2019), while others emphasize user-centered evaluation (e.g., Confalonieri & Alonso-Moral, 2024; Donoso-Guzmán et al., 2023; Holzinger et al., 2020). However, most of these frameworks remain largely theoretical, as they are rarely empirically tested or systematically applied in user-centered research. As a result, it remains unclear which factors are most critical for effective explanations, which factors consistently contribute to positive outcomes, and which do not. Moreover, it is unclear which factors are domain- or use-case–specific and which generalize more broadly across the field of XAI.

Existing reviews of XAI implementations report generally positive, yet mixed, effects of AI explanations on performance and user experience, compared to systems without explanations or without AI support altogether (e.g., Haque et al., 2023; Islam et al., 2022; Johs et al., 2022; Laato et al., 2022; Rong et al., 2024; Schemmer et al., 2022). While such comparisons are useful for establishing whether explanations can influence human-AI interaction, they provide limited insight into which characteristics of explanations are responsible for these effects. If an explanation outperforms a no-explanation condition, it remains unclear whether this is due to the specific explanation implementation or simply the presence of additional information. To gain a clearer understanding of the factors that contribute to effective AI explanations, it is necessary to move beyond comparisons with null conditions and instead directly compare different XAI approaches. Such comparisons should ideally isolate the effects of specific explanation elements while keeping all other variables constant across conditions, so-called matched controls. This approach enables a more precise assessment of which design choices drive differences in outcomes. To date, existing reviews have rarely adopted this comparative perspective, limiting their ability to explain the heterogeneous findings in the literature or to identify which specific design elements contribute to improved performance and user experience. As an exception, Rong et al. (2024) take an initial step by identifying a small subset of 29 studies that directly compare different XAI implementations. Their findings suggest that certain explanation features, such as simplicity, relevance, and alignment with user goals, can influence effectiveness, but also highlight that results remain inconsistent and highly context-dependent. Moreover, their review is restricted to conference publications up to 2022, leaving more recent studies and work published in journals or other venues largely unexplored. As a result, a comprehensive and up-to-date synthesis of studies that systematically compare XAI implementations is still lacking.

In particular, we aim to understand whether explanation effectiveness is primarily determined by the underlying AI or XAI method (as is often the focus in existing studies, such as those summarized in Rong et al., 2024), or whether it depends more on how explanations are presented and how users interact with them.

We conceptualize these influences as a spectrum (See Figure 1), ranging from (X)AI-centered aspects (e.g., model type, explanation algorithm) to human-centered aspects (e.g., presentation format, user expertise, interaction design). While prior research has mostly focused on the technical side of this spectrum, it is still an open question where the most important determinants of effective explanations lie along this spectrum. Identifying the relative importance of these aspects, and whether this varies across use cases, is essential for designing XAI systems that meaningfully support human decision-making.

Therefore, in this paper, we conduct a systematic review of empirical studies that directly compared different implementations of XAI in terms of their effects on human–AI collaboration outcomes. Our review addresses the following research question:

“How do different implementations of explainable Artificial Intelligence (XAI) compare to one another in terms of human-AI interaction outcomes?”

To address this research question, based on a careful comparison of the existing frameworks, we organize the review around four key aspects of XAI explanations. This structure serves two purposes. First, it provides a systematic way to compare the highly diverse explanation implementations found in the literature. Second, it allows us to examine whether explanation effectiveness is primarily determined by technical characteristics of the AI system or by design choices that are closer to the human user.

Existing taxonomies and frameworks distinguish between different types of XAI implementations in various ways (e.g., Donoso-Guzmán et al., 2023; S. Hong & Park, 2025; Miller, 2019; Sheridan et al., 2024). Although these frameworks differ in terminology and emphasis, they share several common elements. Typically, they start with the technical explanation method used to generate the explanation, and subsequently address aspects such as what information should be presented and how it should be presented (i.e., the user interface). Some frameworks additionally consider the interaction between the end user and the system. Across these aspects, the spectrum between AI and human involvement becomes visible again, ranging from the internal workings of the AI system to the user’s interpretation and interaction (as shown in Figure 1).

Building on this shared structure, we follow the core ideas of these frameworks along this spectrum and distinguish four aspects of an explanation: method, content, format, and interaction. These aspects closely resemble the explanation elements described by Donoso-Guzmán et al. (2023), although we adopt a more general terminology.

Explanation method refers to the technical approaches or generation procedures used to produce the explanation (for example, whether the system relies on feature attributions, counterfactual reasoning, or example-based retrieval). This aspect concerns how the explanation is computed.

Explanation content refers to the information that is selected as the explanation, independent of how it is presented. In a feature-based approach, this might be the specific features and their importance scores; in an example-based method, the chosen training examples.

Explanation format concerns the way this content is presented to the user (i.e. the user interface), such as a bar chart, a table of numbers, a highlighted image, or a short textual statement. While the content stays the same, its representation differs.

Lastly, interaction refers to how users are facilitated to engage with the presented explanation. This includes the ways in which users can explore and respond to explanations, for example by requesting additional details, selecting alternative views, or interacting with visual elements.

In the following sections, we examine these four aspects in more detail and outline which explanation elements fall within each aspect. Given the diversity of frameworks, which differ in focus from general AI design to specific XAI mechanisms, this four-part structure helps to organize what might otherwise appear as a fragmented and heterogeneous field. Although this framework provides a useful analytical structure, the boundaries between aspects are not always clear-cut. Explanation elements and comparisons may involve multiple aspects at the same time, and decisions regarding one aspect may shape or depend on decisions in another aspect.

2. Theoretical Background

2.1. Explanation Method

To systematically analyze explanation methods in XAI, prior literature proposes several dimensions along which methods can be classified. A widely adopted taxonomy distinguishes between (i) intrinsic vs. post-hoc explanations, (ii) model-specific vs. model-agnostic methods, and (iii) global vs. local explanations (Molnar, 2025). These dimensions provide a conceptual foundation for organizing explanation methods, which can be further differentiated based on the type of explanation they produce.

Intrinsic (ante-hoc) interpretability refers to models that are inherently transparent by design, such as linear models, decision trees, or rule-based systems. These models enable direct inspection of their internal logic. In contrast, post-hoc methods aim to explain already trained models, often treated as black boxes, without modifying their internal structure. This distinction is important, as post-hoc approaches allow the use of highly complex models, such as deep neural networks, while still providing interpretability (Lipton, 2018).

A second distinction concerns the dependency of explanation methods on the underlying model. Model-specific approaches use internal properties such as gradients, weights, or architectures, whereas model-agnostic methods rely solely on input-output behavior and can therefore be applied universally. Model-agnostic approaches are particularly relevant in practice due to their flexibility and comparability across models (Lundberg & Lee, 2017; Ribeiro et al., 2016).

Finally, explanation methods differ in their scope. Global explanations aim to describe overall model behavior, while local explanations focus on individual predictions. This distinction is especially important when comparing explanation techniques, as some methods inherently provide instance-level insights, whereas others support a more holistic understanding of the model. However, in the categorization used in this paper, the distinction between global and local fits better in the explanation content as it is more about the information provided and does not necessarily depend on the XAI method. Therefore, we will present the scope in more detail in the section about explanation content.

This broad taxonomy by Molnar (2025) provides a conceptual foundation for categorizing explanation methods, which can be further differentiated based on the type of explanation they produce. The type of explanation can be broadly grouped into model-based, feature-based (attribution), and example-based approaches (e.g., Markus et al., 2021; Molnar, 2025). Jin et al. (2022) suggest supplementary information as an additional category, including elements such as model performance metrics and dataset characteristics; however, in the current review we address these under content, as they do not constitute explanation generation methods per se.

Furthermore, counterfactual explanations are often considered a subset of example-based approaches. Counterfactuals focus on identifying minimal changes to the input required to alter a model prediction. The contrastive and action-oriented nature makes them particularly relevant in human-centered and socially meaningful explanation contexts (Miller, 2019; Wachter et al., 2017). Therefore, we argue that counterfactuals constitute a fundamentally different explanatory approach, and hence treat it as a separate category.

2.1.1. Feature-Based (Attribution) Explanations

Feature-based explanations are the most widely used type of XAI methods. They aim to quantify the contribution of input features to model predictions, typically by assigning importance scores to individual features. These approaches are especially prominent in post-hoc, model-agnostic settings due to their flexibility and intuitive outputs.

Prominent attribution methods include LIME, which approximates the model with an interpretable surrogate, SHAP, which leverages concepts from cooperative game theory to distribute prediction contributions among features, and Generalized Additive Models (GAMs), which provide interpretable predictions by modeling the additive contribution of individual features (Caruana et al., 2015; Lundberg & Lee, 2017; Ribeiro et al., 2016). While these methods provide intuitive explanations, they may suffer from instability and rely on assumptions such as feature independence (Alvarez-Melis & Jaakkola, 2018; Lundberg & Lee, 2017).

Beyond individual feature contributions, feature-based approaches also include methods that capture how multiple features combined influence predictions. Techniques such as Partial Dependence Plots (PDPs) and Accumulated Local Effects (ALE) visualize the relationship between features and model outputs, while interaction-aware methods explicitly capture non-additive relationships between features (Apley & Zhu, 2020; Friedman, 2001; Molnar, 2025). Compared to attribution methods, these approaches are less frequently used and tend to be more developer oriented, primarily supporting model developers in understanding overall model behavior rather than providing intuitive explanations for end users. While they offer deeper insights into complex models, they also increase interpretational complexity (Guidotti et al., 2018; Molnar, 2025).

2.1.2. Model-Based Explanations

Model-based explanations achieve interpretability either through inherently transparent models or by approximating complex models with interpretable surrogates. Rather than producing post-hoc attributions, these approaches explain predictions by exposing the structure, parameters, or decision logic of the model itself. As such, they might reveal how input features contribute to outcomes, but in a way that is directly tied to the model’s internal representation rather than computed afterward.

Intrinsically interpretable models include linear and logistic regression, decision trees, and rule-based systems. These models provide global interpretability, as their decision processes can be directly inspected and understood. However, their simplicity may limit predictive performance when applied to complex or high-dimensional data (Lipton, 2018).

To address this limitation, surrogate (post-hoc) models are often employed. A surrogate is an interpretable model trained to approximate the behavior of a black-box model. Such surrogates can be global, capturing overall model behavior, or local, focusing on specific regions of the input space (Molnar, 2025). However, this introduces a fidelity–interpretability trade-off, as simpler models may fail to fully capture complex relationships (Molnar, 2025; Ribeiro et al., 2016).

2.1.3. Example-Based Explanations

Example-based explanations justify model predictions by referencing specific data instances, aligning closely with human reasoning based on analogies and comparisons. Instead of describing abstract feature contributions, these methods explain predictions through concrete examples drawn from the dataset or cases similar to the data.

One common approach is to retrieve similar instances from the dataset that are close to the query point in feature space. These explanations allow users to assess whether a prediction is reasonable by comparing it to known cases. Another approach involves prototype and criticism methods, which summarize datasets using representative examples (prototypes) and identify poorly represented or atypical instances (criticisms), as proposed in the MMD-critic framework (B. Kim et al., 2016).

Another type of example-based explanations are influential instance methods. These methods trace predictions back to training data points that had the greatest impact on the model. Influence functions approximate the effect of individual training samples on predictions, enabling users to detect biases or errors in the training data (Koh & Liang, 2017). These approaches are particularly useful for model debugging and dataset understanding.

2.1.4. Counterfactual Explanations

Counterfactual explanations address “what-if” scenarios by identifying the changes required to change a model’s prediction. They are often grouped under example-based explanation methods, as they are formulated as concrete data instances that are close to a given input. Like other example-based approaches, they rely on comparisons between individual cases to justify model predictions, thereby aligning with human reasoning patterns based on analogy and similarity. In this sense, a counterfactual can be interpreted as a specific type of example that illustrates how the input space relates to the model’s decision boundary.

However, in this review, we treat counterfactual explanations as a separate category due to their fundamentally contrastive and action-oriented nature. Unlike other example-based methods that primarily aim to justify or contextualize a prediction (e.g., by retrieving similar instances or prototypes), counterfactuals explicitly address what the minimal changes are to change a model’s prediction. These minimal changes are typically defined in terms of modifying as few input features as possible, making the smallest necessary adjustments to their values, and ensuring that such changes are feasible or actionable in practice (Wachter et al., 2017). This makes counterfactual explanations intervention-focused rather than purely descriptive. It is closely related to human approaches to explanation, as human explanations often directly answer contrastive questions such as “Why this outcome instead of another?” (Miller, 2019; Wachter et al., 2017).

2.2. Explanation Content

While the explanation method concerns how an explanation is produced, explanation content concerns what information is selected and presented in order to address a user-relevant question. Importantly, the same explanation generation method (e.g., SHAP, counterfactuals, attention weights) can support multiple content types depending on how its output is structured, contextualized, and framed but they may also limit the types of content that can be generated (e.g., certain methods are inherently unsuitable for producing local explanations). The explanation content can be characterized along several dimensions, including the explained object, scope, degree of personalization, level of detail, type of argumentation, and external evidence.

2.2.1. Explained Object 

The explained object refers to the specific aspect of a model or decision that an explanation tries to clarify. It closely aligns with the types of questions users naturally ask when interacting with AI systems. Building on this idea, Lim & Dey (2010) define eight (model-agnostic) objects which can be explained by an explanation method: model inputs, model outputs, what (current output), what if (what is the output for different input values), why (did the model reach this output), why not (different output), how to (produce different output, certainty (how certain is the model). These are highly similar to the questions users might have about specific AI output as defined in the question bank by Liao et al. (2020), as well as the levels of explanations by Mueller et al. (2021). Taken together, explanation content begins with identifying which user question is being addressed.

2.2.2. Scope of Explanations: Local Versus Global

The scope (or granularity) of an explanation refers to whether it describes the behaviour of an AI system at the level of a specific instance (local) or at the level of the entire model (global). In addition, both the evidence (i.e., model-derived information) and the interpretation (i.e., the meaning assigned to that information) can operate at either level (Rizzo et al., 2023). For example, a local interpretation may explain a single attention weight for one prediction, whereas a global interpretation may aggregate attention patterns across many instances to reveal broader trends.

Local explanations focus on individual predictions. They are widely used in research, with about 70% of empirical XAI studies focusing exclusively on local explanations (J. Kim et al., 2024). From a technical standpoint, local explanations can be more accurate than global explanations because the behaviour of a complex model is often simpler (e.g., linear) in a small neighbourhood around a specific data point (Molnar, 2025).

Global explanations, in contrast, describe the overall behaviour of a model. While they provide a more comprehensive view, they are harder to interpret due to model complexity (Molnar, 2025). Global interpretability is often achieved at a modular level (e.g., feature weights or decision rules) rather than for the entire model at once. Common approaches include rule extraction, decision tree approximations, and global feature importance (Molnar, 2025).

2.2.3. Framing

Another aspect of content in XAI concerns the type of framing (or reasoning/argumentation) used in explanations. Explanations can be presented through different forms of reasoning, for example by emphasizing associations between different outcomes, contrasting outcomes or features with plausible alternatives, or highlighting statistical evidence about outcomes, each influencing how users interpret AI decisions. Several studies emphasize the role of argumentative and persuasive framing. Chatti et al. (2024), for example, distinguish between explanations that encourage users to think carefully about the underlying reasoning and explanations that rely more on simple cues such as visual design or heuristics. Alternatively, D. Wang et al. (2019) argue that contrastive explanations, which answer questions such as “Why this outcome instead of another?”, are often more usable than listing all possible causes because they reduce information overload and increase users’ perceived control. In addition, Rosenfeld and Richardson (2019) describe justification as a framing strategy aimed at persuading users that a decision is appropriate or trustworthy, often prioritizing user acceptance over full technical transparency.

2.2.4. Level of Detail

Another important aspect of content in XAI concerns the level of detail in explanations, which determines how much information is presented and must balance informativeness with cognitive simplicity.

A central concept is completeness, referring to the extent to which explanations reflect the full model’s reasoning process and underlying causes (Donoso-Guzmán et al., 2023; Kulesza et al., 2012; Nauta et al., 2023). While more complete explanations can support more accurate mental models, overly detailed explanations may reduce curiosity, engagement, and usability, suggesting that both insufficient and excessive information can be detrimental (Donoso-Guzmán et al., 2023; Kulesza et al., 2012).

As a result, researchers emphasize compactness and parsimony in explanation design. Parsimonious explanations reduce cognitive load by presenting information in a concise form that aligns with human processing limits (Markus et al., 2021; Nauta et al., 2023). Similarly, Molnar (2025) notes that users often prefer brief explanations with only a few key reasons, even for complex systems. This relates to the fidelity paradox, where simplified but plausible explanations can be perceived as more understandable and satisfying than fully complete but complex ones (Molnar, 2025; Rizzo et al., 2023).

Several studies therefore emphasize balancing completeness and compactness in explanation design. While completeness focuses on providing a sufficiently comprehensive account of the factors underlying a model’s output, compactness emphasizes restricting explanations to the most relevant information and presenting it in a concise and cognitively manageable form (Donoso-Guzmán et al., 2023; Kulesza et al., 2012; Markus et al., 2021; Nauta et al., 2023). From this perspective, explanation design involves determining which information is most relevant to disclose, how much detail is necessary for the user and task, and how additional information can be layered or selectively revealed when needed (Haque et al., 2023).

2.2.5. Personalized Explanations

Another important aspect of content in XAI is the personalization of information to the specific user, task, and context. From an explainee-centered perspective, explanations are not static properties of a system, but part of a social process that depends on how they are perceived and understood by the individual receiving them (Miller, 2019).

Personalization can be used to ensure that explanations are clear, coherent, and actionable for a specific user, such that they fit the user’s background and informational needs (cf. context and coherence in Nauta et al., 2023). This can involve adapting both the type and amount of information provided. For example, users may require different forms of explanations depending on whether they ask “why,” “why not,” or “what if” questions, highlighting the importance of adapting explanations to situational goals and informational needs (Lim et al., 2009; Lim & Dey, 2010).

Another dimension of personalization concerns tailoring to user characteristics such as expertise, AI literacy, prior knowledge, mental models, and psychological traits. In a medical context, for instance, a patient may require a simple explanation such as “high blood sugar,” whereas a physician may need a more detailed technical explanation of the underlying biological relationships (Samek & Müller, 2019). More broadly, explanations could be more effective when aligned with users’ existing conceptions, cognitive processes, and technical understanding (e.g., Chazette & Schneider, 2020; Ehsan et al., 2019; D. Wang et al., 2019). Finally, personalization may also account for psychological and cognitive characteristics such as need for cognition, conscientiousness, decision making style, and other individual differences, which can influence how users perceive and interact with explanations (e.g., Conati et al., 2021; Hernandez-Bocanegra & Ziegler, 2021b).

2.2.6. External and Contextual Information

As part of the content level, external or contextual information surrounding an AI system can support users in interpreting explanations, evaluating system reliability, and understanding the boundaries of model applicability. Although this information does not directly explain the internal reasoning process of a model, it provides important supplementary context for decision-making.

One category of contextual information concerns model performance metrics, such as accuracy, precision, recall, ROC curves, calibration scores, and uncertainty estimates. These metrics are described as global supplementary information or “model facts” that communicate overall system quality, limitations, and uncertainty rather than explaining specific prediction logic (Jin et al., 2022; Liao et al., 2021). Performance transparency is important because users often interpret explanations differently depending on their confidence in the model, while communicating calibration and evaluation choices can help users better assess reliability and uncertainty (Adhikari et al., 2022).

Another important category concerns dataset characteristics, including training data distributions, sample sizes, demographic representation, missing values, and excluded datasets. Such information helps users identify potential biases, blind spots, and limitations in model behavior, while also increasing perceptions of transparency and trustworthiness (Haque et al., 2023; Jin et al., 2022; Liao et al., 2020). Questions regarding data provenance, collection methods, and dataset scope are therefore increasingly recognized as important elements within XAI systems.

A third category includes human, social, and domain-based references external to the model itself. External trust signals such as peer reviews, institutional reputation, authority approvals, or endorsements from other users can influence trust and acceptance independently of the technical explanation provided (Chatti et al., 2024; Jin et al., 2022). Similarly, social comparison approaches that present examples of similar users or trusted peers could improve persuasive effectiveness and user engagement (D. Wang et al., 2019).

Finally, human-like and anthropomorphic aspects of AI systems can also shape explanatory perception. Users naturally attribute human characteristics and intentions to algorithmic systems, meaning that conversational agents using human-like communication styles, gestures, or voice interaction may appear more trustworthy and socially acceptable (Haque et al., 2023; Molnar, 2025). In addition, users may accept explanations more readily when they are presented by a perceived authoritative system or institution (M.-Y. Kim et al., 2021).

2.3. Explanation Format

Explanation format refers to the way in which explanatory information is presented to the user. It captures the representational and perceptual characteristics of explanations. Prior research highlights that format is not merely a delivery mechanism but also shapes how explanations are perceived and interpreted by users (e.g., Haque et al., 2023; Rizzo et al., 2023).

A key distinction within explanation format concerns (1) modality, referring to the sensory channel through which information is conveyed, and (2) saliency, referring to how important elements within an explanation are emphasized.

2.3.1. Modality

Modality describes the form in which an explanation is represented and perceived. Explanations can be delivered through different representational formats most commonly textual, graphical, and tabular formats, while less frequently used modalities include images, audio, and video (Adhikari et al., 2022; Haque et al., 2023; Vilone & Longo, 2021). More broadly, explanation modalities can also be combined into multi-modal formats or embedded in richer communication media such as infographics or illustrated text (El-Assady et al., 2019; Vilone & Longo, 2021). Within graphical modalities specifically, different subtypes can be distinguished. Chatti et al. (2024) categorize visual explanations into ten different types, with the node-link diagram and bar chart being most common, followed by the tag cloud, and then by Venn diagram, heatmap, scatterplot, pie chart, treemap, radar chart, and world map.

It needs to be noted that explanation formats are closely linked to the underlying generation method. Jin et al. (2022) explicitly group formats according to explanation generation methods and suggest that each type is associated with a limited set of typical representations. This indicates that the choice of format is constrained by the generation approach rather than being purely a design decision. Closely related, the choice of modality is often constrained by practical considerations such as interface limitations and design standards, reinforcing the need for context-sensitive format selection (Eiband et al., 2018).

2.3.2. Saliency

Saliency refers to highlighting information that is considered most relevant to a model's output. While saliency is often based on attribution methods that estimate the importance of individual input features, it concerns how this information is highlighted and communicated to the user, for example through heatmaps, highlights, color coding, or other visual cues. In this sense, saliency can support explanations about why a prediction was made by drawing attention to influential parts of the input, although the underlying explanation method may vary (Liao et al., 2021). Moreover, saliency-based methods are among the most widely used explanation approaches across domains (Klein et al., 2024).

Saliency is particularly prominent in visual explanation techniques, where importance is encoded through cues such as color, intensity, or spatial focus. Across modalities, saliency can highlight relevant elements such as image regions, words, or text passages through mechanisms like localization or textual highlighting (Nauta et al., 2023; D. Wang et al., 2019). In practice, saliency is most commonly realized through heatmaps or color-coded visualizations. Saliency maps and heatmaps are widely identified as dominant formats for feature attribution, using color scales to represent fine-grained importance scores (Islam et al., 2022; Jin et al., 2022).

2.4. Explanation Interaction

Explanation interaction refers to explanation as an interactive process between user and AI system rather than a static output. It focuses on how users could engage with explanations by asking questions, filtering information, or exploring results, and it enables explanations to become adaptive, user-centered, and dialogical. Interactivity can be divided into different types of interaction techniques: selective, mutable, and dialogic interaction (Bertrand et al., 2023).

Selective or on-demand explanations refer to systems where users request explanations only when needed, rather than receiving all information automatically, often called “progressive disclosure” (Springer & Whittaker, 2020). On-demand disclosure could reduce cognitive overload and might give users more control when additional contextual information is revealed (Haque et al., 2023; Laato et al., 2022).

Mutable explanations refer to interaction modes in which users actively modify inputs, parameters, or conditions of the AI system to observe how predictions or explanations change. These interactions correspond to common user questions, such as “what-if” scenarios (Liao et al., 2020) or the effects of adjusting system parameters (Sipos et al., 2023), enabling users to actively explore the model’s behavior. This form of interaction aligns closely with the concept of controllability, which captures the extent to which users can influence or “correct” the system, and is identified as a key quality dimension in XAI evaluation frameworks (Donoso-Guzmán et al., 2023; Nauta et al., 2023).

Dialogic or conversational interactivity refers to explanation systems that engage users in a back-and-forth dialogue, where explanations are shaped by user questions and follow-up interactions. Liao et al. (2021) describe this as a question-driven grounding process, where explanations are continuously adapted to user queries in order to reduce the gap in understanding through iterative clarification.

The importance of interaction in explanation is reflected in a broader view within the literature that conceptualizes explanation as a social process grounded in continuous (interactive) information exchange between an explainer and an explainee (Miller, 2019). In their framework of explanation levels, M.-Y. Kim et al. (2021) describe the most advanced level as interactive and explainee-aware, enabling users to iteratively refine their questions through conversational explanations and receive tailored clarifications. Interactivity therefore functions as an advanced and important explanatory capability that supports identifying knowledge gaps, enabling clarification, and building user trust through dynamic exchange (M.-Y. Kim et al., 2021). Additionally, explanation is an iterative process that is particularly important during verification phases, where user feedback plays a critical role in refining understanding and explanation systems (El-Assady et al., 2019). The importance of interaction is further reflected in XAI evaluation frameworks, where dimensions such as controllability and adaptivity are commonly used to assess the quality of explanatory systems (Cortiñas-Lorenzo et al., 2025; Donoso-Guzmán et al., 2023).

3. Method

We conducted a systematic literature review of empirical studies that compared different implementations of XAI with each other. We aimed to create an overview of studies that involve human participants in the evaluation of XAI and that compare multiple explanation approaches within the same experimental context.

3.1. Literature Search Strategy

We performed our systematic literature review following the PRISMA guidelines for systematic reviews (Shamseer et al., 2015), to identify user studies that compare XAI implementations on end-user evaluations. The protocol for this systematic review was preregistered on Protocols.io (Ronckers et al., 2024). We systematically searched the electronic databases Web of Science, Scopus, IEEE Xplore, and ACM Digital Library on May 14th, 2024. The search query contained words related to Explainable AI and user studies (See Table 1 for the full search strategy). The search was limited to title, abstract, and keywords for most databases, and “all metadata” for IEEE Xplore. Because database interfaces and search functionalities differed, the exact queries were adapted for each database. Detailed search strings for all databases are provided in Appendix A – Search Queries.

The original protocol also included an additional literature saturation step following the initial screening, involving backward reference searching and the identification of related publications using AI tools. However, given the large number of records retained after screening, the inclusion of this step would have required a substantial expansion of the screening and eligibility assessment process, and was therefore not pursued further.

3.2. Selection Criteria

For the literature selection, we defined a set of eligibility criteria based on the different aspects of the research question. The inclusion and exclusion criteria are outlined in Table 2. We included studies on “explainable” or “interpretable” AI, for any application domain. Recommender systems were included as AI systems as well. Studies should compare different XAI implementations with each other, which could be a comparison between XAI generation methods (e.g., feature-based, example-based, counterfactuals, etc.), content (e.g. level of detail, local versus global, etc.), formats (e.g., modality, saliency, etc.) or interaction (e.g. on-demand possibilities, chatbots, etc.).

We excluded comparisons between different black-box AI models without different XAI implementations and comparisons between different XAI models whose differences were not noticeable for end users. The XAI implementations could be applied in any domain or use cases and comparisons between application domains were included as well.

Studies were included if they measured some form of human-AI interaction outcomes. This could be subjective experiences of the users, such as trust, likeability, preference, or objective measures, such as effectiveness, performance of the human-AI collaboration, decision time, or behavioral reliance. There was no restriction on the year of publication (with the final date being May 14th, 2024).

3.3. Results Screening

For the screening process, we used online software Rayyan (Ouzzani et al., 2016) for both the title-and-abstract screening phase and the full-text screening phase, making use of the platform’s blind-mode functionality, in which reviewers performed independent assessments while being blinded to one another’s screening decisions to reduce bias. The initial search yielded 4,035 articles. Search results were imported into Rayyan (Ouzzani et al., 2016), where duplicate removal was first performed automatically. Initially, Rayyan automatically resolved duplicates with a similarity score of 97% or higher. Because closer inspection revealed that a substantial number of duplicates remained, we added an additional duplicate screening step that had not been specified in the protocol, in which Rayyan resolved duplicates with a similarity score of 95% or matching titles and DOI numbers. Remaining duplicates were removed manually, resulting in a final set of 3,034 unique articles for screening.

Although Rayyan provides relevance scores indicating how well records may fit the review topic, all records were screened manually. After a pilot screening of 50 articles to refine the screening criteria and establish consensus among the authors (MR, RC, and CS), the first author (MR) screened the full set of 3,034 titles and abstracts of all records. Studies that clearly did not meet the eligibility criteria were excluded by MR. Studies for which eligibility was uncertain were additionally screened independently and in blind mode by RS (220 papers). Papers marked as “include” by RS were included, while remaining doubtful cases were resolved through discussion. In total, 285 articles remained for full-text screening.

Because Rayyan did not support automatic retrieval of full texts, full-text articles were searched for, downloaded, and uploaded manually for the second screening phase. Full-text screening was conducted by MR. Doubtful cases were resolved through discussion amongst all authors (MR, RC, and CS). This process resulted in an included sample of 206 papers. An overview of the complete screening procedure is provided in Figure 2.

3.4. Quality Assessment

For the quality assessment, all 206 included papers were evaluated using the Mixed Methods Appraisal Tool (MMAT; Q. N. Hong et al., 2018). The MMAT includes both general criteria regarding the research question and data quality, as well as method-specific criteria for qualitative, quantitative, randomized controlled, and mixed-methods studies. In this review, the MMAT was used as an overall quality appraisal tool to assess the methodological rigor and reporting quality of the included studies, rather than to evaluate a specific type of bias. For each paper, we assessed all categories applicable to the study design. In some cases, multiple categories were scored within a single paper because the paper included multiple study types or methodological approaches.

To obtain an overall quality score, we calculated the average across all rated MMAT items, using the following coding scheme: 2 = “Yes” (criterion clearly met and reported), 1 = “Can’t tell” (criterion could have been met, but reporting was unclear), and 0 = “No” (criterion not met or not reported). The general MMAT item regarding clear research questions (S1) was excluded from the overall score calculation, as this criterion appeared overly strict for the included literature. Many papers did not formulate an explicit research question, but their research aims and direction were generally clear from the introduction and study description.

The original protocol specified that only studies with extremely low overall scores would be excluded from the review, with the threshold to be determined after assessment (Ronckers et al., 2024). In the final process, we excluded studies with an average MMAT score of 1 or lower across the assessed criteria from further analysis. This threshold reflected studies in which the majority of criteria were rated as either “No” or “Can’t tell,” indicating substantial concerns regarding methodological transparency or study quality. Applying this criterion resulted in the exclusion of 45 papers and resulted in a final set of 161 papers for data extraction.

3.5. Data Extraction

Data from the included studies were extracted manually by MR, MV, and RLN, who divided the papers among themselves. A final consistency check across all papers was subsequently conducted by MR. As an additional independent check, RC and CS did a final consistency check on the papers extracted by MR.

All extracted data were managed using Covidence software (Covidence Systematic Review Software, 2025) and recorded in a predefined data extraction form, which was refined by MR, RC, and CS after pilot extraction of 25 papers, leading to adjustments in item wording, order, and category definitions. The extracted data included detailed study characteristics such as research aims, participant demographics, and study context. In addition, extensive information was collected on the XAI interventions, including a classification according to explanation method, content, format, and interaction characteristics. Data were also extracted on study design features, including sample size, experimental setup, comparison conditions, and the underlying AI and user tasks. Outcome-related data included the human-AI interaction constructs assessed, measurement instruments, all reported effects on XAI comparisons, and the direction of findings across experimental conditions (see Supplementary Materials 1[1] for the full data extraction). Following extraction, MR synthesized the data in a table format, identifying for each paper which XAI conditions performed better across specific outcome measures (see Supplementary Materials 2¹). Additionally, an overview of the visuals of XAI implementations used in each paper was created (see Supplementary Materials 3¹).

Originally, the protocol specified that, if relevant information was missing in the papers or supplementary materials, study authors would be contacted (up to three attempts). However, this step was not undertaken due to the large set of included studies. Missing information was generally limited and did not substantially affect data extraction. Moreover, studies with insufficient reporting quality were excluded during the quality assessment, reducing the likelihood that missing information influenced the review findings.

4. Overview of the Field

Our systematic review showcases a wide variety of studies. The field of XAI is broad in all senses: many different application domains, use cases, implementations, and presentation formats coexist. In addition, user studies are conducted in diverse ways with different methods and measurements. To contextualize the heterogeneity in findings, we first provide an overview of the included studies, focusing on application domains, participant populations, and methodological approaches. This overview contextualizes the subsequent differences in outcomes across studies and highlights structural characteristics of the field that may shape the effectiveness of XAI. A complete overview of all extracted study characteristics, including sample sizes, participant characteristics, outcome measures, measurement instruments, and study results, is provided in Supplementary Materials 1.

Figure 3 presents the distribution of publications over time. Especially in recent years, the number of user-centered XAI studies that compare different explanation implementations has grown substantially, indicating that this line of research is rapidly gaining attention. This growth mirrors the general growth of XAI research, but also suggests a more recent shift toward empirical, comparative, and human-centered evaluations. With regard to publication venues, the majority of included studies were published in conference proceedings (60.2%) rather than journals (39.8%). This aligns with the prominence of conferences in fields such as computer science and human–computer interaction, as well as the rapidly growing nature of the field.

5. Results

Understanding how different explanation designs influence human–AI interaction is central to evaluating the practical value of XAI. While prior work often compares explanations to a no-explanation baseline, far fewer studies directly contrast different kinds of explanations with one another. The studies in our review do exactly that: they compare variations in how explanations are generated, what content they present, and how this content is formatted and communicated to users. These works do not necessarily compare explanations against the absence of explanations. Instead, by comparing explanations with one another, they move beyond the question of whether explanations help at all, and instead examine which design choices meaningfully affect user experience and performance.

For each of the four components of explanations (method, content, format, and interaction), we summarize what types of comparisons the included studies made, how these design choices influence human–AI interaction, and where results converge or diverge. In doing so, we highlight which explanations tend to make a difference (compared to other explanations), where effects remain inconsistent, and which design choices appear most promising for supporting effective and meaningful human–AI collaboration.

Across the reviewed studies, one overarching pattern becomes clear: there is no universally best explanation. Most comparisons reveal mixed or non-significant differences, and when differences do appear, they are often dependent on the use case, task demands, or user characteristics. At the same time, certain explanation factors are more influential than others. Our overview suggests that the explanation format (such as graphical layouts) and interactivity features tend to influence human-AI interaction the strongest. The content of explanations shows moderate influence, yet again with substantial variability across studies. In contrast, the underlying technical XAI method used to generate an explanation tends to have the least consistent impact on user outcomes.

Interpreting the presented findings requires caution. Many of the included studies compared different explanations, but they rarely varied only one factor in the explanation’s design; instead, often multiple design aspects changed at once, making it difficult to pinpoint which elements actually drive the results. Although we summarize overarching patterns, individual findings are shaped by many contextual factors, such as the specific explanation design, domain or use case, task difficulty, user characteristics, and study setup. These differences make it challenging to compare studies directly, and certain nuances may be lost when synthesizing findings at a high level. In addition, qualitative descriptors such as “a few,” “many,” or “the majority” are used throughout this section instead of exact percentages. Because individual studies often reported multiple outcome measures, with some yielding significant and others non-significant results, precise percentages could be misleading. Readers should therefore keep these limitations in mind when interpreting our conclusions. For those who want to examine the exact conditions under which particular results were obtained, all study-level details can be found in the Supplementary Materials 1.

5.1. Limited Impact of Explanation Method

Across the studies that compared different XAI generation methods, such as feature-based, example-based, and counterfactual explanations, there is no clear evidence that one method consistently outperforms the others. About two thirds of the (measured) outcomes did not show significant differences between methods.

The vast majority of studies comparing example-based and feature-based explanations found no significant differences. This holds for understanding (Bove et al., 2022; Dominguez et al., 2020; Gentile et al., 2023), trust or reliance (Du et al., 2022; Gentile et al., 2023; Tsai et al., 2021; B. Wang et al., 2024), effectiveness (Gedikli et al., 2014), interaction or response time (Dominguez et al., 2020; Gedikli et al., 2014; Gentile et al., 2023; Linder et al., 2021), satisfaction (Gedikli et al., 2014; Guesmi et al., 2021; Tsai et al., 2021), transparency (Gedikli et al., 2014; Tsai et al., 2021), cognitive load (Gentile et al., 2023; Tsai et al., 2021; B. Wang et al., 2024), and performance (Linder et al., 2021; Tsai et al., 2021) or general user ratings (B. Wang et al., 2024).

The same pattern appears in studies comparing counterfactual and feature-based explanations. These studies also most often report non-significant results for understanding (Melsion et al., 2023; Riveiro & Thill, 2021), trust or reliance (de Brito Duarte et al., 2023; Jakubik et al., 2023; Larasati et al., 2020; Lim et al., 2009; Melsion et al., 2023; Scharowski et al., 2023; Warren et al., 2022, 2023; Woodcock et al., 2021), confidence (Celar & Byrne, 2023), satisfaction (Jansen et al., 2024; Riveiro & Thill, 2021; Warren et al., 2022, 2023), performance (Celar & Byrne, 2023; Ibrahim et al., 2023; Jakubik et al., 2023; Lim et al., 2009; Warren et al., 2022, 2023), personal attachment (Larasati et al., 2020), completeness (Riveiro & Thill, 2021), and general user experience (Jansen et al., 2024). For counterfactual explanations compared to example-based, there were no significant differences for agreement, performance, and helpfulness (Perlmutter et al., 2024).

Studies that compared example-based, feature-based, and counterfactual explanations (all three in the same study) reported no significant differences in understanding (Shulner-Tal et al., 2022; X. Wang & Yin, 2021, 2022), trust or reliance (Naiseh et al., 2023; A. Silva et al., 2023; X. Wang & Yin, 2021, 2022), interaction or response time (Cau et al., 2023; A. Silva et al., 2023), performance (Cau et al., 2023; Gombolay et al., 2024; Naiseh et al., 2023; A. Silva et al., 2023), explainability (Gombolay et al., 2024; A. Silva et al., 2023), and fairness (Shulner-Tal et al., 2022).

Although most studies found no significant differences between XAI methods, about one third of the individual results report that certain XAI methods work better than others. However, these findings are inconsistent and do not indicate a universally best XAI method, with methods outperforming others in some cases but underperforming in others.

Compared to feature-based explanations, counterfactual explanations showed higher user understanding (Le et al., 2020; Maruf et al., 2023; Naiseh et al., 2023; Perlmutter et al., 2024; Schulze-Weddige & Zylowski, 2022), accuracy (Celar & Byrne, 2023; Le et al., 2020; Mertes et al., 2022), trust/reliance (Maruf et al., 2023; Mertes et al., 2022; Perlmutter et al., 2024) satisfaction, self-efficacy, and confidence (Mertes et al., 2022), user experience (Naiseh et al., 2023), perceived helpfulness (Celar & Byrne, 2023), completeness, and preference (Maruf et al., 2023), and intuitiveness (Le et al., 2020). However, these effects often depended on additional factors: Schulze-Weddige & Zylowski (2022) found them only for LIME and not for SHAP, Celar & Byrne (2023) observed them only when the AI was correct and effects varied by domain familiarity, and Maruf et al. (2023) reported them only in the nursery scenario, not the “telecom” scenario. Combining counterfactuals with examples reduced trust, persuasiveness, and perceived quality compared to examples alone or examples accompanied by text (Gates et al., 2023). Rule-based counterfactuals improved objective understanding relative to example-based counterfactuals, though subjective understanding, performance, reliance, and usability did not differ (van der Waa et al., 2021).

Additionally, example-based explanations led to better understanding (Naiseh et al., 2023), accuracy (V. Chen et al., 2023), perceived competence (B. Wang et al., 2024), agreement (Cau et al., 2023) than feature-based explanations. Qualitative studies suggest that example-based explanations have higher persuasiveness, transparency, accuracy, and satisfaction (Lu et al., 2023) than feature-based explanations and align more closely with intuitive reasoning, support learning through examples, and provide clearer signals of AI unreliability (V. Chen et al., 2023) compared to feature-based explanations that were seen as visually overwhelming and sometimes led to inappropriate reliance (V. Chen et al., 2023).

The effectiveness of an explanation method can also depend on its content. Global example-based explanations improved trust and transparency when explaining the input, whereas feature-based explanations were more effective for explaining the output of the AI system (Guesmi et al., 2021).

On the other hand, feature-based explanations showed higher efficiency (Guesmi et al., 2021), understandability (B. Wang et al., 2024) and higher preference (S. S. Y. Kim et al., 2023) than example-based explanations, and better subjective understanding, perceived quality, and indirect trust (but not direct trust) (de Brito Duarte et al., 2023) compared to counterfactuals. When the AI output was correct, feature-based explanations were associated with higher satisfaction, completeness, and understanding than counterfactual explanations, but no significant differences were observed when the AI output was incorrect (Riveiro & Thill, 2021). Similarly, qualitative findings showed that factual, feature-based explanations were preferred over counterfactual explanations when the output was similar to expectations, but both combined was preferred when the output was unexpected (Riveiro & Thill, 2022).

Overall, no clear distinction between subjective and objective outcomes emerged. Advantages reported for specific explanation methods were observed for both types of measures, but these effects were highly inconsistent and context-dependent. However, the positive effects reported for feature-based explanations were predominantly subjective in nature, including outcomes such as perceived understanding, trust, and preference. In contrast, the positive effects of counterfactual and example-based explanations were observed across both subjective and objective measures, including improvements in perceived evaluations as well as understanding and accuracy.

Specific Feature-based methods: LIME, SHAP, and GAM

Because LIME and SHAP are two of the most widely used feature-based explanation methods, several studies have compared them directly to examine whether they differ in how users interpret and evaluate explanations. Across those studies, the two methods generally perform similarly, and almost all outcomes show no significant differences. We see this for understanding (Jalali et al., 2023; Knapič et al., 2021; Schulze-Weddige & Zylowski, 2022; X. Wang & Yin, 2021, 2022), predictability (Jalali et al., 2023), performance (Knapič et al., 2021; Malhi et al., 2020), and trust (X. Wang & Yin, 2021, 2022). Another feature-based method commonly used is GAM: GAM explanations outperform SHAP on accuracy, cognitive load, and confidence in understanding the explanation, but SHAP results in higher confidence in reasonable explanations when explanations or AI output is wrong (Kaur et al., 2020).

Other explanation generation methods are typically less researched in the literature. We highlight some of the results: decision trees were less preferred and induced the longest processing time compared to feature maps or natural language, but decision trees caused the fewest consecutive mistakes after incorrect advice; feature maps led to the most inappropriate compliance (A. Silva et al., 2024). White box regression explanations caused more prediction errors than both local or global LIME explanations, but trust, usefulness, and time showed no significant differences between the explanation types (Burkart et al., 2021). A visual map explanation reduced cognitive load and improved satisfaction for participants with low AI literacy compared to SHAP, decision trees, and counterfactuals (Jansen et al., 2024).

5.2. Mixed Effects of Explanation Content

The findings across explanation content choices show that what an explanation communicates can influence human–AI interaction, but often in nuanced and context-dependent ways. Some factors, such as the amount of information, show clearer patterns, whereas others, such as confidence scores, local or global, personalization, argumentation and framing, or explanations with human-oriented cues, have more mixed or limited effects across contexts.

5.2.1. Amount of Information

The findings on the amount of information in an explanation suggest a tension between what users prefer and what helps them perform best. On the subjective level, participants frequently want more information: richer, more complete, or more interactive explanations tend to produce higher understanding, satisfaction, trust, and perceived usefulness. Yet, objective outcomes tell a different story: performance and cognitive load studies show that very detailed explanations can hinder users, and that a balanced, intermediate level of information often works best. Overall, the findings point toward an explanation design that avoids both extremes: users appreciate more detail, but their performance benefits most from an optimal, moderately complex explanation, rather than offering maximal or minimal information.

Impact on Subjective Experience

Subjective evaluations generally improved with more, or optimally balanced, information. Providing more examples improved subjective trust (Perlmutter et al., 2024), understanding (Bove et al., 2023; Perlmutter et al., 2024), user interest (Zhang et al., 2022) and satisfaction (Bove et al., 2023 when comparative analysis is added as well; Zhang et al., 2022 for textual explanations) compared to one single example. Similarly, providing more complete information was preferred (Ehsan et al., 2019; Maruf et al., 2023) over shorter basic explanations, and resulted in higher ratings for user experience such as fairness, usefulness, completeness, ease of use, trust/acceptance, and satisfaction (Dodge et al., 2019; Hermann et al., 2023; Khurana et al., 2021; Larasati et al., 2020; Maruf et al., 2023; Naveed et al., 2018; Rago et al., 2021). Finally, combining separate explanation types (sometimes with different modalities) also resulted in higher trust, interpretability, ease of use, perceived confidence, and persuasiveness (Radensky et al., 2022; Szymanski et al., 2021; Weitz et al., 2021; Wibowo et al., 2018) and qualitative results suggest that combined explanations are preferred and more persuasive and useful compared to single, simple explanations (Du et al., 2022; Heuer & Breiter, 2020; Sato et al., 2018, 2019). Some studies mentioned a more optimal level of information: although no statistical tests were reported in Chatti et al. (2022), intermediate levels of detail appeared to result in higher satisfaction, trust, and scrutability compared to basic and advanced explanations. However, these findings depended on personality characteristics, such as need for cognition, visualization familiarity, and personal innovativeness (Chatti et al., 2022; Guesmi et al., 2022). In the study by Heuer & Breiter (2020), participants emphasized the value of exposing inner mechanisms and providing more information, while also noting a trade-off between informativeness and potential overload. Additionally, mixed findings also suggest this optimum: in Kleinerman et al. (2018) explanations with less information were rated higher on relevance, satisfaction, and competence, but more information was rated higher on trust and acceptance. Only the study by Abdul et al. (2020) suggests that less information can lead to more positive subjective outcomes: less information was easier to understand.

One study pointed to preferences for simpler (non-hybrid) formats: Martijn et al. (2022) found that brief explanations (e.g., preference-fit score or example-based) were preferred over more information-rich multimodal combinations that included feature-based, example-based, and scatter-plot components. Additionally, while there were no overall preference differences between bar charts and more complex multimodal explanations, users low in openness preferred bar-chart-only formats, whereas users high in openness sometimes preferred the more detailed, multimodal information.

At the same time, about half of the reported findings did not show significant differences in subjective ratings across explanation sizes or detail levels. This includes outcomes such as fairness (Barile et al., 2021, 2024; Goyal et al., 2024), consensus / agreement (Barile et al., 2021, 2024; Perlmutter et al., 2024), satisfaction (Barile et al., 2021, 2024; Guesmi et al., 2021; Qu et al., 2021; Stepin et al., 2022; Tsai & Brusilovsky, 2019b; Wibowo et al., 2018), trust / reliance / confidence (Conijn et al., 2023; de Brito Duarte et al., 2023; Ehsan et al., 2019; Hou et al., 2024; Qu et al., 2021; Ribes et al., 2021; Stepin et al., 2022; Wibowo et al., 2018), motivation (Conijn et al., 2023), usefulness / usability / helpfulness (Cruz et al., 2022; Perlmutter et al., 2024; Qu et al., 2021; Radensky et al., 2022; Ribes et al., 2021; Robbemond et al., 2022), human-likeness (Ehsan et al., 2019), explanation quality (Wilkinson et al., 2021), persuasiveness (Guesmi et al., 2021; Tsai & Brusilovsky, 2019b), informativeness (Stepin et al., 2022), completeness (Sivaprasad et al., 2024), understanding (Qu et al., 2021; Radensky et al., 2022; Ribes et al., 2021; Sivaprasad et al., 2024), perceived competence (Ribes et al., 2021), transparency (Wibowo et al., 2018), scrutability (Guesmi et al., 2021), and bias perception (Hou et al., 2024). Tsai & Brusilovsky (2019b) likewise reported mostly non-significant results for subjective experience, with only a single comparison showing an effect: adding a bar chart to a Venn Word Cloud improved transparency, scrutability, trust, effectiveness, and perceived performance.

Impact on Cognitive Load and Time

Several studies indicate that providing more information in explanations tends to increase both interaction time and cognitive load. More extensive interfaces resulted in higher (objective) cognitive load when having more graphs (Abdul et al., 2020) or explanation elements (Andjelkovic et al., 2016; Sanneman & Shah, 2022). Similarly, adding more elements to the explanations increased response time (Linder et al., 2021) and interaction time (Radensky et al., 2022) compared to only one explanation element. In contrast, two studies reported an opposite pattern: participants experienced higher cognitive load (Dodge et al., 2021) and longer perceived time (Szymanski et al., 2021) when presented with less information or types of explanations. However, about half of the studies did not find any significant influences of the amount of information for interaction time (Jesus et al., 2021; Stepin et al., 2022; Stites et al., 2021; Tran et al., 2019, 2020; Tsai & Brusilovsky, 2019b; Wibowo et al., 2018) or cognitive load (Robertson et al., 2021; Tsai & Brusilovsky, 2019b).

Impact on Performance

In our sample, richer explanations were typically associated with better human–AI performance. Combined explanations resulted in higher (subjective) performance than explanations with one element alone (Das & Chernova, 2020; Gentile et al., 2023; Qu et al., 2021). Simply adding more information or more graphs also resulted in higher performance, understanding, and improved skills to highlight inconsistencies and logic errors in the decision path (Abdul et al., 2020; Dodge et al., 2021; T. Liu et al., 2023; Mccalmone et al., 2022; Sanneman & Shah, 2022). Similarly, the results from Ibrahim et al. (2023) suggested that providing more information resulted in higher accuracy, although this was not statistically tested. Interestingly, one study found the opposite result, where performance was better with less information (Sivaprasad et al., 2024), but mental models did change more when interacting with more extensive explanations. Still, about half of the studies did not report any significant findings of the amount of information on performance (Andjelkovic et al., 2016; Goyal et al., 2024; Linder et al., 2021; Perlmutter et al., 2024; Radensky et al., 2022; Ribes et al., 2021; Robertson et al., 2021; Stepin et al., 2022; Stites et al., 2021; Tsai & Brusilovsky, 2019b; Wibowo et al., 2018).

These mixed findings may indicate that explanation effectiveness depends on finding a balance in the level of detail provided, rather than simply offering more or less information. Intermediate levels of information resulted in better performance compared to the simplest and most extensive explanations (Pierson et al., 2024), although only for difficult tasks. Similar results were found by Chatti et al. (2022), although not statistically tested.

5.2.2. Confidence and Probability Scores

Confidence or probability scores constitute the most minimal type of explanatory information, as they provide only a numerical indication of model certainty without offering insight into how a decision was reached. Although such scores are often assumed to make AI decisions clearer or more trustworthy compared to other types of explanations, the empirical evidence offers little support for this assumption. Across studies, these numerical indicators generally do not improve how people understand, trust, or use AI systems. A large majority works report no significant effects of confidence scores on accuracy or performance (Brachman et al., 2023; Buçinca et al., 2021; Gombolay et al., 2024; Linder et al., 2021; Qu et al., 2021), understanding (Brachman et al., 2023), trust (Buçinca et al., 2021; Conijn et al., 2023; Mehrotra et al., 2024), preference (Buçinca et al., 2021), cognitive load (Buçinca et al., 2021; Qu et al., 2021), motivation (Conijn et al., 2023), satisfaction and effectiveness (Gedikli et al., 2014), time (Gedikli et al., 2014; Linder et al., 2021), difficulty (Qu et al., 2021), willingness to pay (Ben David et al., 2021), and usefulness (Mehrotra et al., 2024), compared to other types of explanations.

When effects do appear, they are mixed. A few studies show benefits, such as improved adoption or agreement (Ben David et al., 2021; Linder et al., 2021) or higher preference (Waa et al., 2020) compared to other types of explanations such as feature-based explanations. Others find that confidence scores compare unfavorably to richer or more interpretable explanation formats: they reduce transparency and effectiveness (Gedikli et al., 2014), perform worse on explainability measures compared to decision trees, case-based or feature importance (Gombolay et al., 2024), or result in lower satisfaction and usability compared to highlighted explanations (Qu et al., 2021). Notably, the few significant effects that were reported primarily concerned subjective outcomes, whereas objective outcomes were almost exclusively null findings.

5.2.3. Local versus Global Explanations

While the majority of studies find no significant differences between local and global explanations, the few significant results observed tentatively point to a slight advantage for local explanations. Non-significant differences were found across a wide range of outcome measures including performance (Burkart et al., 2021; Huber et al., 2021; Naiseh et al., 2023; Radensky et al., 2022), trust (Ben David et al., 2021; Burkart et al., 2021; Naiseh et al., 2023; Radensky et al., 2022; B. Wang et al., 2024; X. Wang & Yin, 2021, 2022), understanding (Naiseh et al., 2023; Radensky et al., 2022; Sivaprasad et al., 2024; X. Wang & Yin, 2021, 2022), usefulness (Burkart et al., 2021; Naiseh et al., 2023; Radensky et al., 2022; Sivaprasad et al., 2024) and several other user experience measures (Ben David et al., 2021; Guesmi et al., 2021; Septon et al., 2023; B. Wang et al., 2024).

When significant differences do appear, they tend to favor local explanations over global. Local explanations result in higher trust (Ben David et al., 2021), understanding (Septon et al., 2023), efficiency (Guesmi et al., 2021), performance and shorter explanation length (Evirgen et al., 2024; Sivaprasad et al., 2024), and had more impact on participants’ decision changes under high AI confidence (H. Chen et al., 2024). Specifically for positively phrased recommendations, local explanations resulted in higher perceived fairness and understanding (Shulner-Tal et al., 2022) Qualitative findings also suggest a preference for local explanations: users preferred local why explanations and perceived them as more effective compared to global how explanations, although the global explanations were perceived as more transparent and trustworthy (Guesmi et al., 2023).

Evidence supporting global explanations is more limited and mixed. Global explanations could be more useful (Radensky et al., 2022), help users detect inconsistencies better (Dodge et al., 2021), and reduce cognitive load (Dodge et al., 2021; B. Wang et al., 2024) relative to local explanations.

The mixed effects might depend on the content or design of the explanation: Guesmi et al. (2021) reported that global explanations resulted in higher trust and transparency when the goal was to explain system input, but when explaining system output, local explanations scored higher. B. Wang et al. (2024) reported that global explanations resulted in better understandability compared to some local explanations but not all, and for competence, global explanations scored in between two local explanation types.

5.2.4. Personalized Explanations

Even though personalized explanations are often proposed as a way to make AI systems more meaningful to individual users, findings suggest that personalized explanations offer limited practical value and do not reliably improve user outcomes compared to non-personalized explanations. In the included literature, personalized explanations typically present information that is specifically relevant to an individual user, for example by highlighting particular features or by adapting the explanation based on prior interactions, preferences, or behavioral history. The vast majority of included studies report no significant effects of personalization at all. They found null results of personalization on trust (Hernandez-Bocanegra & Ziegler, 2020, 2021b; Tintarev & Masthoff, 2008; Tsai et al., 2021), reliance (Matarese et al., 2023), transparency (Hernandez-Bocanegra & Ziegler, 2020, 2021b; Lu et al., 2023; Tsai et al., 2021), effectiveness (Gedikli et al., 2014; Hernandez-Bocanegra & Ziegler, 2020, 2021b; Tintarev & Masthoff, 2008), subjective quality (Hernandez-Bocanegra & Ziegler, 2020, 2021b), satisfaction (Gedikli et al., 2011; Hernandez-Bocanegra & Ziegler, 2020, 2021b; Lu et al., 2023; Matarese et al., 2023; Naveed et al., 2018; Tsai et al., 2021), system acceptance, completeness, and intention to use (Naveed et al., 2018), as well as on time (Gedikli et al., 2011, 2014), accuracy (Gedikli et al., 2011; Lu et al., 2023), performance and user experience (Matarese et al., 2023; Tsai et al., 2021), persuasiveness (Lu et al., 2023; Tsukuda & Goto, 2020), learning and mental load (Tsai et al., 2021).

The few studies that found significant effects of personalized explanations, reported findings in different directions. Participants have higher satisfaction (Gedikli et al., 2014; Tintarev & Masthoff, 2008) and transparency (Gedikli et al., 2014 for tag clouds) with personalized explanations compared to non-personalized explanations. Additionally, qualitative findings found that personalized explanations were the most appreciated feature within an interactive explanation setting (Guesmi et al., 2024) and resulted in higher confidence, transparency, and help to communicate, but this was not replicated in the quantitative results (Tsai et al., 2021). On the other hand, personalized information seems less easy to understand, useful, and persuasive than information based on other users or content (Hernandez-Bocanegra & Ziegler, 2020, 2021b; Kouki et al., 2019, 2020).

One study tried to find the most optimal way to personalize the explanation design: A. Silva et al. (2024) found that balancing users’ preferred explanation modality (feature importance, language, or decision trees) with optimal task performance resulted in higher user preference and lower inappropriate compliance compared to randomly chosen modalities or maximizing modality preference. No other significant differences were observed, including when explanations were personalized based only on task performance or exclusively language explanations.

Overall, personalization showed little impact on either subjective or objective outcomes, but the few positive effects that were observed were largely confined to subjective measures such as satisfaction, transparency, confidence, and user appreciation rather than objective measures such as accuracy, performance, or time.

5.2.5. Framing

Framing choices can influence how explanations are perceived but the specific effects depend strongly on the type of framing used and the situation in which it is applied. Several studies indicate that framing information in a more “argumentative” or “contrastive” way can improve user outcomes compared to a more descriptive statement. Using argumentative facts (keywords with words like “therefore”) were more persuasive than simple factual keywords, although fully argumentative sentences were the least persuasive (Zanker & Schoberegger, 2014). For invalid statements, questioning framings improved accuracy and informativeness compared to causal framings, whereas for valid statements, causal framings were more informative (Danry et al., 2023). “General” explanations performed worse than contrastive, truthful, and thorough explanations for both trust and personal attachment (Larasati et al., 2020). And finally, presenting one-sided information resulted in more user-interest than two-sided information, but satisfaction did not differ (Zhang et al., 2022). The specific way in which a contrastive explanation was framed appeared to have limited impact. Counterfactual explanations (how an outcome could have been different in the past) and prefactual explanations (how it could be different in the future) did not differ in terms of perceived accuracy or helpfulness (X. Dai et al., 2022).

Also other framing than “argumentative” aspects were investigated: positive framing improved understanding (Hadash et al., 2022; Kuhl et al., 2023), performance, and subjective experience (Kuhl et al., 2023) compared to negative framing. The perceived power of an explanation also appeared to influence user perceptions. High-power explanations increased users' sense of control, although they did not affect perceptions of system-related factors (Ha et al., 2022). Likewise, framing a system as highly autonomous reduced cognitive load and increased social presence and faith in the system compared to low-autonomy framing, but no differences were found in interaction fluency, confidence, or trust (B. Wang et al., 2024). Additionally, providing context-relevant information led to higher satisfaction compared to context-irrelevant details (Zhang et al., 2022) and using keywords that are familiar to the target user resulted in higher confidence than unfamiliar keywords (Evirgen et al., 2024). Similarly, using language that matched the user's wording resulted in better understanding than using less aligned language (Srivastava et al., 2023).

Framing effects also depend on the perceived reliability and correctness of the presented information. Papenmeier et al. (2022) shows that explanations framed with higher accuracy cues result in higher objective trust than medium or antagonistic framings. However, the effects differ between subjective and objective outcomes: while faithful explanations (i.e., correctly reflecting the model) improve objective trust compared to random information under high accuracy conditions, this advantage does not consistently translate to subjective trust. In some cases, subjective trust was even higher without explanations than with faithful or random explanations, indicating that users do not always respond to explanation quality in a straightforward way.

5.2.6. Human-Oriented Explanation Content

Two further factors studied in explanation design are the use of social information and the degree to which explanations appear human-like. The literature on these aspects shows mixed outcomes for social information and more promising patterns for human-like presentation.

Social and User-based Information

Evidence on whether explanations based on other users or “social” information are helpful show no consistent pattern: social or user-based information can sometimes help, sometimes harm, and often makes no measurable difference. About half of the individual findings report no significant differences when explanations are based on information of other humans rather than system-based or generic information, including for understanding (Brachman et al., 2023), trust and reliance (Matarese et al., 2023; A. Silva et al., 2023; Woodcock et al., 2021), time (A. Silva et al., 2023), performance and accuracy (Matarese et al., 2023; A. Silva et al., 2023), compliance, explainability (A. Silva et al., 2023), user experience (Matarese et al., 2023), perceived quality (J. Dai et al., 2023), and satisfaction (Matarese et al., 2023). About a quarter of the findings show benefits of social information, such as higher accuracy (Brachman et al., 2023), greater persuasiveness (Í. Silva et al., 2024; Tsukuda & Goto, 2020), better effectiveness (Í. Silva et al., 2024), and better user experience (Hasanah & Kusumo, 2024) compared to explanations based on generic information. Lu et al. (2023) also reported that peer-based and similar-user information appeared better for transparency, persuasiveness, accuracy, and satisfaction than generic explanations, though no statistical testing was performed. Other findings indicate the opposite: generic information explanations increase persuasiveness, usefulness (Sato et al., 2018, 2019), fairness, understanding, (Shulner-Tal et al., 2022), and satisfaction (J. Dai et al., 2023), and could be more preferred and useful (Hasanah & Kusumo, 2024) compared to social-based information. There is also some evidence that too much social information may not be helpful: Donkers et al. (2020) found that selected, aspect-based reviews performed better than large sets of user reviews.

Human-like Explanations

A recurring pattern across studies is that explanations presented in a more human-like way are generally preferred or lead to better outcomes than more technical or machine-like formats. Textual explanations framed like a human increased awareness of harm, perceived responsibility (Hindennach et al., 2024), understanding, user experience (Mukhtar et al., 2023), and increased usefulness ratings for lay-users, but not for AI experts (Cruz et al., 2022) compared to computer-based explanations. Additionally, textual highlights based on human strategies increased perceived quality and lowered processing time compared to computer-based highlights (Pafla et al., 2024). Similarly, adding human-like modalities (like audio or virtual agents) increased trust (Weitz et al., 2021). Finally, explanations perceived with higher human-agency increased trust in the system, which subsequently enhanced choice confidence (W. Liu & Wang, 2025). In this study, explanations attributed to an AI assistant or similar users were perceived as having greater human agency than those attributed to an algorithm (W. Liu & Wang, 2025).

At the same time, a majority of studies found no significant differences between human-like and machine-like formats on bias perception (Hou et al., 2024), trust or reliance (Ben David et al., 2021; Hou et al., 2024; Pafla et al., 2024), usefulness (Cruz et al., 2022), responsibility (Hindennach et al., 2024), correctness (Hindennach et al., 2024), satisfaction (Ben David et al., 2021; Pafla et al., 2024), performance (Pafla et al., 2024), cognitive load (Pafla et al., 2024).

Overall, the reported benefits of human-like explanations were more evident for subjective outcomes, such as perceived understanding, usefulness, and user experience, than for objective outcomes, where effects were largely absent.

5.3. More Promising Impact of Explanation Format

In this section, we turn to the explanation format, where we find yet more consistent and significant effects. In other words, how an explanation is presented seems to matter at least as much as, and in many cases more than, the underlying method or the specific content shown. Findings suggest there is no single format or design choice that works best in every context. Instead, the effectiveness of an explanation seems to depend on how well it organizes and structures information and how clearly it communicates key insights. Transparent formats that structure underlying information, often graphical or tabular, frequently support faster and more accurate understanding, but simpler or more constrained designs can be preferable when cognitive load needs to remain low.

5.3.1. Modality

Because modalities differ in how they structure information and how easily users can extract information and form an overview of what is going on, comparing them offers insight into which formats work well under which conditions. The following subsections compare these modalities systematically, beginning with textual and tabular formats and then moving on to visualizations, image-based, audio, and video. Overall, the evidence across modalities suggests that no single modality is universally best. Instead, users benefit most when explanations present information in a structured, clear, and cognitively manageable way, regardless of whether the medium is textual, tabular, or graphical.

Text Versus Tabular Formats

More than two thirds of the studies report no significant differences between text and tabular formats for performance (Gombolay et al., 2024; A. Silva et al., 2023), trust (Gates et al., 2023; Hernandez-Bocanegra & Ziegler, 2021a; A. Silva et al., 2023; X. Wang & Yin, 2021, 2022), effectiveness (Hernandez-Bocanegra & Ziegler, 2021a), time (A. Silva et al., 2023), subjective quality (Gates et al., 2023; Hernandez-Bocanegra & Ziegler, 2021a), perceived explainability (Gombolay et al., 2024; A. Silva et al., 2023), and other user experience measures (Gates et al., 2023; Hernandez-Bocanegra & Ziegler, 2021a).

The few significant findings tend to favor more organized formats, such as word clouds, interactive lists, or tables over plain natural language. Tabular explanations score higher on subjective quality (Hernandez-Bocanegra & Ziegler, 2023)and might be more preferred (Rago et al., 2021) than (conversational) natural language. Additionally, structured word clouds are preferred over basic word clouds (Tsai & Brusilovsky, 2019a) and have higher satisfaction and accuracy and shorter interaction time compared to tabular lists (Gedikli et al., 2014).

Graphical Versus Textual Formats

Overall, the included studies comparing graphical and textual explanation formats show that graphical explanations generally outperform plain text, if significant differences exist. This happens especially when graphical explanations offer a clear and quickly understandable overview.

Graphical representations lead to higher understanding (de Brito Duarte et al., 2023; Kumar et al., 2022), subjective trust, and quality ratings (de Brito Duarte et al., 2023), helpfulness (Heuer & Breiter, 2020), effectiveness and lower time (Gedikli et al., 2014), and higher reliance for high confidence-models (X. Wang & Yin, 2021, 2022) compared to textual alternatives. Qualitative studies support this positive effect of graphical formats: participants generally preferred graphical representations over text-based ones (Szymanski et al., 2022; Tsai & Brusilovsky, 2019a) and participants noted that these graphs provided a high level of transparency and insight into how their input contributed to the outcome (Szymanski et al., 2022).

Only a few studies show the opposite pattern, where textual explanations produce better understanding and more positive ratings (Mukhtar et al., 2023) and higher performance (Ribeiro et al., 2018; Szymanski et al., 2021), less interaction time (Ribeiro et al., 2018) than graphical ones. Closer inspection of these cases suggests that this tends to occur when the graphical representations are unclear or difficult to interpret.

Differences might depend on the specific design of the graphical formats, as some graphs are more understandable than text and others less (Schulze-Weddige & Zylowski, 2022), on the correctness of the advice (A. Silva et al., 2024), or it might depend on personal characteristics of the user, such as need for cognition or personal innovativeness (Chatti et al., 2022) or domain knowledge (Bhattacharya et al., 2023). In addition, Bhattacharya et al. (2023) noted that graphical representations supported quick understanding, textual annotations improved clarity, and many participants indicated a tendency to use multiple explanation aspects together.

Contrasting the promising findings of graphical formats compared to text, more than half of the studies find no significant differences at all for performance (A. Silva et al., 2023), trust (de Brito Duarte et al., 2023; Hernandez-Bocanegra & Ziegler, 2021a; Robbemond et al., 2022; A. Silva et al., 2023; X. Wang & Yin, 2021, 2022), understanding (Dominguez et al., 2020; X. Wang & Yin, 2021, 2022), interaction time (Dominguez et al., 2020; A. Silva et al., 2023), satisfaction (Gedikli et al., 2014; Guesmi et al., 2021; Hernandez-Bocanegra & Ziegler, 2021a), and several other user experience measures (Gedikli et al., 2014; Guesmi et al., 2021; Hernandez-Bocanegra & Ziegler, 2021a; Kumar et al., 2022; Robbemond et al., 2022; A. Silva et al., 2023; Szymanski et al., 2021).

Graphical Versus Tabular Formats

When comparing graphical formats to more organized textual formats such as tables, the differences become much less pronounced. In many studies, once text is presented in a structured way, the clear advantage that graphical representations usually have over plain text seems to disappear. A substantial number of studies show no significant differences between graphical and tabular formats, including performance (Naiseh et al., 2023; Stites et al., 2021), trust (Du et al., 2022; Hernandez-Bocanegra & Ziegler, 2021b, 2021a; X. Wang & Yin, 2021, 2022), understanding (Le et al., 2023; X. Wang & Yin, 2021, 2022), cognitive load (Jansen et al., 2024), effectiveness (Gedikli et al., 2014; Hernandez-Bocanegra & Ziegler, 2021b, 2021a), interaction time (Gedikli et al., 2014), satisfaction (Gedikli et al., 2014; Guesmi et al., 2021; Hernandez-Bocanegra & Ziegler, 2021b, 2021a; Jansen et al., 2024), reliability (Naiseh et al., 2023), persuasiveness (Guesmi et al., 2021; Hernandez-Bocanegra & Ziegler, 2021b), and other user experience measures (Gedikli et al., 2014; Guesmi et al., 2021; Hernandez-Bocanegra & Ziegler, 2021b, 2021a; Jansen et al., 2024; Naiseh et al., 2023; Tsai & Brusilovsky, 2019a).

Still, a few studies do report significant differences, although the findings point in different directions. In some cases, graphical explanations are more preferred (Du et al., 2022), result in higher reliance for high-confidence models (X. Wang & Yin, 2021, 2022), result in higher trust and satisfaction, specifically for one type of data (Le et al., 2023) compared to tabular explanations. In other cases, tabular explanations are more transparent, trusted, and efficient than decision-tree graphs (Guesmi et al., 2021), reach higher accuracy (V. Chen et al., 2023), and they may lead to higher satisfaction, trust, scrutability, persuasion, efficiency, and effectiveness compared to graphical formats (Chatti et al., 2022; Ha & Kim, 2023), although these last results were not tested statistically.

Images Versus Text-based Formats

Using images, icons, or other visuals that are not graphs, is not inherently better or worse than providing information textually, but their value depends on how clearly and meaningfully they support the explanation. About two thirds of the studies report no significant differences when visuals are compared to textual explanations for performance (Morrison et al., 2024), time (Cau et al., 2023), user experience (Guo et al., 2022; Jansen et al., 2024), satisfaction (Jansen et al., 2024; Wibowo et al., 2018), trust or reliance (Hou et al., 2024; Wibowo et al., 2018), bias perception (Hou et al., 2024), and effectiveness, efficiency, or transparency (Wibowo et al., 2018). Additionally, adding an image as part of the (textual) explanations did not improve trust and perceived usefulness (Mehrotra et al., 2024).

However, when visuals genuinely help structure information, they may outperform text. Showing a visual map results in lower cognitive load than a tabular explanation (Jansen et al., 2024), natural language alone takes longer to process than designs incorporating thumb-icons (Wibowo et al., 2018), and visual versions offer higher usefulness, transparency, and trust compared with textual versions with highlights (Khurana et al., 2021). At the same time, visuals can mislead users when they are unclear or difficult to interpret, as explanations using images of birds caused higher levels of deception than explanations presented in natural language (Morrison et al., 2024).

This could depend on the AI certainty: under high AI certainty, images lead to higher performance and text to higher user agreement with AI (Cau et al., 2023), but under high uncertainty, this pattern reverses such that text leads to better performance and image to higher agreement with AI.

Images Versus Graphical Formats

Comparing images with graphs appears more difficult, because the effectiveness of each seems to depend strongly on the application, and several studies find non-significant differences in understandability or time (Dominguez et al., 2020), trust (Ghods & Cook, 2022), or satisfaction (Jansen et al., 2024).

Yet when images provide a clear visual overview fitted to the task, they can outperform graphs in the right context. For time series data, pictures seem to have the best performance, highest understandability, and shortest completion time compared to graphs (Ghods & Cook, 2022). And a well designed static visual map, resembling an image, leads to a lower cognitive load and better user experience than graphs, and higher satisfaction for users with low AI knowledge (Jansen et al., 2024).

Types of Graphical Formats

Graphical representations sometimes work better than other modalities and sometimes do not, and the diversity of graphical formats makes this even harder to evaluate. Findings suggest that the effectiveness depends less on the specific graph chosen and more on how understandable it visualizes the underlying structure: graphical representations work well when they structure information clearly, avoid unnecessary complexity, and support the task.

Graphical representations that structure the underlying data instead of showing the raw data reduce time (Arendt et al., 2020), cognitive load (Abdul et al., 2020), and improve recall (Arendt et al., 2020), although other measures do not necessarily improve, such as precision (Arendt et al., 2020) and subjective understanding (Abdul et al., 2020). Qualitative work also points toward the value of graphs that structure information where participants mentioned that some types of graphical formats were preferred because they helped identifying relevance and comparing numbers more easily (Tsai & Brusilovsky, 2019a). The positive effect of structure is further supported by studies showing no significant differences when graphical representations share similar information and structure, including comparisons between pie and bar charts across transparency, satisfaction, effectiveness, and time (Gedikli et al., 2014), SHAP versus decision-tree explanations for satisfaction, cognitive load, and user experience (Jansen et al., 2024), different bar-chart variants for understanding, reliability, user experience, and performance (Naiseh et al., 2023), and rule-based graphs and decision trees for user experience (Vilone & Longo, 2022). Similarly, LIME and SHAP graphs, despite conceptual differences, often yield similar outcomes as well with no show significant differences between those graphs for understandability or predictability (Jalali et al., 2023; Schulze-Weddige & Zylowski, 2022), and bias-detection performance (Malhi et al., 2020). Again, the design matters more: a simple SHAP bar plot produces better understanding than a SHAP decision plot, SHAP force plot, or LIME, suggesting that structure and clarity drive effectiveness (Schulze-Weddige & Zylowski, 2022).

Yet not all structured graphs are necessarily superior and ease of understandability matters too: Yang et al. (2020) finds that a simple grid is perceived as more helpful than a decision tree, and the tree is more helpful than a more complex connected graph. Then again, for a rose-plot comparison, appropriate trust is higher for trees and graphs than it is for grids, whereas objective trust and confidence do not differ significantly across designs.

The effectiveness of graphical formats may also depend on the outcome measure considered. For example, tables and counterfactual bar-graphs both score higher on subjective understanding than feature-based graphs, while counterfactual graphs score highest on user experience among technically competent users (Naiseh et al., 2023).

At the same time, the “best” graphical representation seems to depend on the task. In an educational advising context, advisors found a slider interface slightly more usable, while a rose plot was easier for understanding student skills; the slider worked especially well when the goal was to see the impact of a specific feature (Scheers & De Laet, 2021). Tintarev et al. (2018) likewise reports that bar charts are better than chord diagrams (or radial networks) for understanding and confidence when explaining popularity, but the opposite pattern emerges for illustrating blind spots.

Audio Explanations

Most studies show that adding audio does not improve explanations compared to text or visuals. Across many outcome measures, audio makes no significant difference: trust (Avetisyan et al., 2022; Robbemond et al., 2022; B. Wang et al., 2024), credibility or usability (Robbemond et al., 2022), interaction fluency, user experience, perceived confidence (B. Wang et al., 2024), and satisfaction (Zhang et al., 2022). In fact, audio can sometimes reduce user interest compared to textual explanations (Zhang et al., 2022).

Still, audio can be helpful when added to existing explanations in time-sensitive contexts or in contexts where channels besides the auditory channel are overloaded: it increases situational awareness in a driving scenario (Avetisyan et al., 2022), reduces mental load and improves understandability with time-series data (B. Wang et al., 2024), and increases trust in a game scenario (Weitz et al., 2021).

Video Explanations

Similar to audio, videos do not appear to improve explanations. Videos result in lower performance (Pierson et al., 2024) and lower understanding (Septon et al., 2023) than a non-video alternative, and there are no significant differences in satisfaction (Septon et al., 2023) or understandability (Singla et al., 2023). The content of the video might influence the effectiveness of video explanations: a video explaining a series of counterfactual images had higher ratings for justification of the decision and helpfulness but lower perceived quality compared to a video explaining only two images (Singla et al., 2023).

5.3.2. Saliency

With saliency we refer to highlighting important information, either in text (for example, highlighting, bolding, or coloring words) or in images (for example, heatmaps, contours, bounding boxes, cut-outs, or dots). The evidence base differs for textual highlights and image-based saliency.

Textual Highlights

Overall, the effectiveness of textual highlights is mixed, with a large majority of the studies showing non-significant findings for performance (Cau et al., 2023; Gedikli et al., 2011; Pafla et al., 2024; Qu et al., 2021), effectiveness (Gedikli et al., 2014), time (Cau et al., 2023; Gedikli et al., 2011, 2014; Pafla et al., 2024), cognitive load (Pafla et al., 2024), trust (Gaube et al., 2023; Pafla et al., 2024), satisfaction (Gedikli et al., 2011, 2014; Pafla et al., 2024), and perceived quality (Donkers et al., 2020; Pafla et al., 2024).

Several studies show that textual highlighting could help in some cases: it improved perceived transparency (Gedikli et al., 2014), confidence, understanding, usability, satisfaction, and reduced perceived difficulty and workload (Qu et al., 2021) relative to no highlights. Additionally, textual highlights resulted in higher agreement, but this depended on the certainty of the AI (Cau et al., 2023) and, through mediated effects, it resulted in higher perceived quality (Donkers et al., 2020) compared to no highlighting. Finally, Pafla et al. (2024) found that with human-highlighted text or answer-only highlighting, participants spent less time on the task and they rated it as higher quality and more helpful than with computer-highlighted text.

Image-Based Saliency

For image-based saliency, the overall picture is also mixed, with again a number of non-significant results for performance (Das & Chernova, 2020; Huber et al., 2021; Maehigashi et al., 2023b, 2023a; Pierson et al., 2024), acceptance (Maehigashi et al., 2023b, 2023a), understandability, justification, quality, or helpfulness (Singla et al., 2023).

Saliency often performs worse than explanations without saliency cues: adding a bounding box around an image region increases users’ misclassification of images compared with no box (Kenny et al., 2023). Additionally, saliency cues result in lower accuracy, satisfaction, trust, self-efficacy, and confidence (Mertes et al., 2022), justification and helpfulness but received higher quality ratings (Singla et al., 2023) compared to counterfactual explanations. Also compared to example-based explanations, heatmaps were less preferred, while they did result in higher confidence (Evirgen et al., 2024). In a driving scenario, image-based or decision tree highlights, resulted in lower preference, more inappropriate compliance, more mistakes, and longer decision times compared to natural language based explanations (A. Silva et al., 2024). In a qualitative study, participants mentioned that heatmap-based explanations felt more unsatisfying because they do not answer the “why” question, but just the “where to look” question (S. S. Y. Kim et al., 2023). On the other side, only one study found a positive effect: a bounding box improved accuracy and perceived quality for non-experts (Gaube et al., 2023).

One study reported more mixed findings: under high uncertainty, highlights in abductive explanations led to higher agreement than no highlights, although they reduced performance, and under low uncertainty, this pattern reverses (Cau et al., 2023).

Several studies compare different ways of visualizing saliency but the results are diverse and do not point in a single best design. Some evidence suggests that saliency directly placed over the image leads to higher confidence and lower cognitive load rather than shown separately (Hudon et al., 2021; Karran et al., 2022). The provided detail with saliency also gave mixed results. Less detail seems to work better in some studies: more detailed heatmaps could result in higher cognitive load (Hudon et al., 2021; Karran et al., 2022) and longer interaction time (Knapič et al., 2021), and detailed outlines led to lower satisfaction whereas simple cut-outs help users better recognize correct decisions (Knapič et al., 2021). Other studies show the opposite: outlines or dense heatmaps supported higher confidence in no-overlay conditions (Hudon et al., 2021; Karran et al., 2022) and boxes and masks get worse quality ratings, than heatmaps and contours (Ammeling et al., 2023). Additionally, it is best to use traditional heatmap color schemes while focusing on smaller, low-level features to support higher accuracy (Famiglini et al., 2024).

At the same time, a similar number of studies report no significant differences between saliency designs: considering understanding or correctness for outlines versus heatmaps or cut-outs (Knapič et al., 2021), confidence in adjacent versus non-adjacent highlights (Karran et al., 2022), acceptance and accuracy for heatmap explanations with or without pointing cues (Maehigashi et al., 2023a, 2023b), or performance and agreement across techniques such as LIME cut-outs, Grad-CAM outlines, and feature-based visualizations (Morrison et al., 2023).

5.4. Strong Effects of Explanation Interaction

Interaction possibilities such as changing the input/output, conversational interfaces, and on-demand access seem to have a strong effect on human-AI interaction. In practice, interactivity is often implemented through interface elements such as buttons or links that allow users to navigate different aspects of an explanation, controls to adjust which features or inputs are considered, or interactive visualizations that enable users to explore and manipulate the model output. Although not uniformly beneficial across all contexts, these interactive and user-controlled elements tend to yield clearer and more consistent improvements in subjective experience, perceived usefulness, and understanding than many of the changes in explanation method or content.

Studies that vary levels of interactivity overwhelmingly report positive effects: subjective experiences (Andjelkovic et al., 2016; Hernandez-Bocanegra & Ziegler, 2021a; Robrecht et al., 2023), quality ratings (L. Chen et al., 2019; Hernandez-Bocanegra & Ziegler, 2021a, 2023; Yeh et al., 2023), persuasiveness (L. Chen et al., 2019), usefulness (Khurana et al., 2021), and transparency (Khurana et al., 2021), and users mentioned more engagement (Prabhudesai et al., 2023). Interactivity also strengthens users’ trust and confidence in the system (L. Chen et al., 2019; Hernandez-Bocanegra & Ziegler, 2021a; Khurana et al., 2021; Yeh et al., 2023) and enhances their understanding of the AI and its explanations (Robrecht et al., 2023; Schmude et al., 2023; Septon et al., 2023, qualitative), as well as the interpretability (Bhattacharya et al., 2023) and it creates more effective interactions (Hernandez-Bocanegra & Ziegler, 2021a; Yeh et al., 2023). Users also report more perceived control when interacting with the system (Andjelkovic et al., 2016; Guo et al., 2022; Prabhudesai et al., 2023).

Some downsides do emerge: interactivity can increase time (Gedikli et al., 2011) and users mentioned higher cognitive load (Prabhudesai et al., 2023), although L. Chen et al. (2019) find the opposite, i.e., with interactivity requiring less time compared to static textual explanations.

Only a few studies report no effects of interactivity for satisfaction (Septon et al., 2023), understanding (Lim et al., 2009; Yeh et al., 2023), trust (Lim et al., 2009), and no clear effects in a qualitative study as well (Schmude et al., 2023). Very few studies report negative effects of interactivity: it lowered one measure (out of multiple) of efficiency (Robrecht et al., 2023), satisfaction (Gedikli et al., 2011), and accuracy (Gedikli et al., 2011) and less interaction possibilities were preferred over more interaction by low-music sophistication participants (Martijn et al., 2022).

Overall, the positive effects of interactivity were observed predominantly for subjective outcomes, such as satisfaction, trust, perceived usefulness, and understanding, although some benefits were also reported for objective outcomes. In contrast, the limited null and negative findings were more frequently associated with objective outcomes, including efficiency, accuracy, time, and cognitive load.

5.4.1. Conversational Explanations

A specific form of interactivity is conversational interaction, such as chatbot-based explanations, which have become increasingly common with LLMs. Only a few studies compare conversational explanations with other formats, and the results are mixed. Some studies suggest that a graphical user interface has higher quality ratings (Hernandez-Bocanegra & Ziegler, 2023) and preference (Rago et al., 2021) compared to a conversational interface. Compared to plain textual explanations, one study suggests that textual is preferred over conversational (Rago et al., 2021), whereas another study suggests that conversational explanations support better understanding (Schmude et al., 2023).

5.4.2. On-Demand Explanations

Whether explanations should be shown up front or offered on demand mainly depends on how much information the explanation contains. For short explanations, presenting them up front results in higher preference (Martijn et al., 2022; Zang & Jeon, 2022), higher perceived competence and intelligence (Pu & Chen, 2007; Zang & Jeon, 2022), lower cognitive/work load (Pu & Chen, 2007; Zang & Jeon, 2022), and stronger intention to return (Pu & Chen, 2007) compared to demand explanations. One reason for the advantage of up-front explanations may be that adding on-demand options increases system complexity even when the amount of information is small (Buçinca et al., 2021).

In contrast, when explanations are long or when users already receive substantial information at the start, on-demand formats increased satisfaction (Bove et al., 2022), improved performance with incorrect AI predictions (Buçinca et al., 2021), and was preferred (Guesmi et al., 2024) and perceived as higher in warmth (Zang & Jeon, 2022) over up-front explanations.

Most significant effects are on subjective experiences, while objective measures more often show no differences for understanding (Bove et al., 2022), performance or accuracy (Buçinca et al., 2021; Zang & Jeon, 2022), trust (Buçinca et al., 2021; Zang & Jeon, 2022), preference (Buçinca et al., 2021), cognitive load (Buçinca et al., 2021; Zang & Jeon, 2022), user experience (Dominguez et al., 2020), time (Pu & Chen, 2007), and effort (Zang & Jeon, 2022).

6. Discussion

In this study, we take a comparative perspective on explainable AI by systematically reviewing studies that evaluate multiple explanation implementations side by side. This approach allows observed differences in outcomes to be attributed more directly to variations in explanation design. A central methodological decision was to restrict the analysis to studies that directly compare two or more explanation types within the same experimental setup, thereby ensuring matched conditions across comparisons. This contrasts with much of the earlier XAI literature, which often contrasts explanations with a no-explanation baseline (or no clear baseline at all). By focusing on explanation-versus-explanation contrasts, we could examine what changes when one XAI method is replaced by another, rather than simply whether explanations have an effect at all. To systematically capture these design differences, explanation implementations were analyzed across four dimensions: method, content, format, and interaction. This framework enabled a structured comparison of how variations in explanation design influence user outcomes.

The first key finding and most notable pattern in the reviewed literature is that, even when different explanation implementations are compared directly, consistent effects remain remarkably scarce. Earlier reviews have already shown that the impact of XAI on outcomes such as trust, understanding, and performance is often mixed and context-dependent (Haque et al., 2023; Laato et al., 2022; Rong et al., 2024; Schemmer et al., 2022). Our review extends this observation by focusing specifically on explanation-versus-explanation comparisons. Despite providing a more controlled basis for evaluating explanation design choices, these studies rarely converge on clear conclusions about what works best. Most comparisons yield mixed or non-significant effects, and no single explanation type consistently outperforms others. At the same time, this does not imply that explanations do not matter. Rather, the findings suggest that explanation effectiveness is highly dependent on contextual factors and the alignment between explanation design, user needs, and task characteristics.

A second key finding of this review is that explanation aspects become more influential as they move closer to the user. When organizing the results along the spectrum from more technical (AI-centered) to more human-centered aspects (see Figure 1, page 3), a consistent pattern becomes visible: explanation characteristics that shape how users interact with, experience, and interpret explanations generally have a stronger impact on outcomes than characteristics related to the underlying explanation algorithm.

Among these factors, explanation interaction appears most influential. Features that allow users to explore, query, modify, or request additional information consistently show the clearest and most consistent positive effects across studies. These findings align with conceptualizations of explanation as an interactive and social process rather than a static transfer of information (Miller, 2019), as well as recent frameworks that emphasize controllability, adaptivity, and dialogue as central qualities of effective XAI systems (Bertrand et al., 2023; Donoso-Guzmán et al., 2023; M.-Y. Kim et al., 2021). Together, these findings point toward a broader shift from static explanation outputs to more dynamic, adaptive, and dialogical forms of XAI.

Following interaction, explanation format also plays an important role. Structured and well-organized representations, such as clear visualizations, tables, or hybrid formats, tend to support understanding and efficiency, while unstructured or overly complex formats can hinder interpretation. Notably, the advantage of visual explanations over text largely disappears when textual information is presented in a similarly structured way. These results suggest that explanations should not be seen merely as outputs of an AI system, but as user interfaces that organize and communicate information in ways that align with human cognitive processes. This finding is consistent with earlier work highlighting explanations as communication and interface design problems rather than purely technical artefacts (Eiband et al., 2018; Haque et al., 2023; D. Wang et al., 2019).

In contrast, explanation content shows more moderate and context-dependent effects. Choices such as the amount of information, framing, or personalization can influence outcomes, but findings are mixed and the effectiveness depends strongly on the task, user, and setting. Elements that align more closely with human reasoning, such as human-like explanations or the inclusion of socially meaningful information, tend to show somewhat more consistent positive effects on perceived understanding, usefulness, and user experience. This is consistent with prior work emphasizing the importance of tailoring explanations to users’ informational needs, mental models, and contextual circumstances (Liao et al., 2020; Lim & Dey, 2010; Rosenfeld & Richardson, 2019).

Finally, the explanation method, such as feature-based, example-based, or counterfactual explanations, shows the weakest and least consistent effects. Most studies comparing these methods find no systematic differences, suggesting that differences in explanation algorithms alone are often insufficient to explain variation in end-user outcomes. This supports earlier critiques that the XAI field may place disproportionate emphasis on algorithmic differences, while the user-facing aspects of explanations receive less attention (Cabitza et al., 2023; Miller, 2019; Rong et al., 2024).

A third key finding of this review is that many XAI design choices involve trade-offs rather than clear improvements. A key example is the tension between fidelity and usability: more detailed explanations increase satisfaction and perceived understanding, but often lead to higher cognitive load and lower performance, with users performing best at moderate levels of complexity. This reflects broader discussions on balancing completeness and compactness in explanation design (Kulesza et al., 2012; Markus et al., 2021; Nauta et al., 2023). Similarly, visual explanations can be effective, but only when they are clear and well-structured; otherwise, they may hinder understanding. Interactivity also illustrates this balance: it often improves engagement and perceived usefulness, yet increases time investment and system complexity, reflecting broader discussions of controllability and progressive disclosure in interactive XAI systems (Donoso-Guzmán et al., 2023; Springer & Whittaker, 2020).

A fourth key finding of this review is that subjective and objective do not always align. Improvements in subjective experience do not necessarily translate into improvements in objective performance. Explanations that increase trust, satisfaction, or perceived understanding often have limited, mixed, or even opposing effects on decision quality, performance, appropriate reliance, or cognitive load. This observation aligns with previous reviews. For example, Rong et al. (2024) note that subjective understanding often improves with explanations, whereas objective metrics such as decision accuracy, trust calibration, and usability show mixed or no improvement. More generally, the misalignment between perceived and actual utility has increasingly been recognized as a key challenge in XAI evaluation (Haque et al., 2023; Islam et al., 2022; Johs et al., 2022). Our findings extend this observation by showing that this disconnect persists even when different explanation implementations are compared directly, rather than only when explanations are compared against a no-explanation baseline.

Taken together, these findings suggest a shift in perspective for XAI. The effectiveness of explanations appears to depend less on the specific algorithm used to generate them, and more on how they are communicated, structured, and aligned with human needs. This implies that XAI should not be approached primarily as a technical problem, but increasingly as a design problem of human–AI interaction. Small, user-facing design choices, such as enabling interaction, choosing an appropriate format, or balancing information complexity, can have a larger impact than differences between explanation algorithms themselves. While calls for a more user-centered perspective on XAI are not new (Laato et al., 2022; Miller, 2019; D. Wang et al., 2019), our findings provide evidence that this shift in focus is warranted. The most consistent effects emerge from user-facing design choices rather than from differences in the underlying explanation algorithms themselves.

6.1. Implications for Research in XAI

Building on the results of our review, we identify a set of key contributions to the field of explainable AI. These contributions reflect both what can be learned from existing comparative studies and what is needed to advance future research in a more systematic and impactful way.

First, research should move away from focusing on the question of “which XAI generation method is best,” and instead approach XAI as a user-centered design problem. Although the importance of user-centered XAI has been acknowledged for some time (Laato et al., 2022; Miller, 2019; D. Wang et al., 2019), our review demonstrates why this perspective deserves greater attention. Across a large body of direct explanation comparisons, the most robust effects were associated with user-facing design characteristics rather than with the underlying explanation methods. Based on this, we suggest that future research should explicitly treat XAI as a user-centered design problem. This means focusing more on how explanations are presented and used, such as format, interaction, and presentation, and clearly justifying these design choices in relation to the user needs and task context.

Second, explanations should be designed for fit rather than general superiority. Much of the existing literature implicitly treats explanations as universally “better” or “worse,” aiming to identify a single optimal design. However, our results highlight substantial variability in effects, suggesting that the value of an explanation is strongly context-dependent and shaped by the task, the user, the domain, and their interactions. This suggests that the effectiveness of XAI lies not in optimizing a single explanation format, but in matching explanations to their specific context of use. Consequently, researchers should move beyond main-effect comparisons and adopt research designs that explicitly model interaction effects (e.g., explanation × task) and moderation (e.g., explanation × user expertise), focusing on when and for whom particular explanations are effective, rather than if they are effective at all.

A third implication of our findings is the need to clearly separate subjective experience from objective performance. Across the reviewed studies, explanations that improved trust, satisfaction, or perceived understanding often showed limited or inconsistent effects on performance, appropriate reliance, cognitive load, or decision quality. This pattern was observed across multiple explanation types and design choices, suggesting that positive user perceptions should not automatically be interpreted as evidence of improved human-AI collaboration. Therefore, subjective improvement should not be treated as the only measure for effective human–AI collaboration. Instead, evaluations should systematically include both subjective and objective performance measures and explicitly report their (mis)alignment.

The fourth implication concerns the need for more targeted comparative research designs, adopting more structured matched conditions. Many existing studies compare XAI with no-XAI conditions or vary multiple explanation features simultaneously. Such designs are valuable for identifying explanations that work well in practice, but they make it difficult, if not impossible, to isolate which specific aspects of an explanation impact the observed effects. Our review therefore highlights that XAI research could benefit from more systematic and controlled comparative approaches that isolate single explanatory factors. Researchers should prioritize tightly matched conditions, clearly specify what differs between them, and vary explanation elements in a structured way. Such study designs are essential to disentangle the mechanisms underlying explanation effectiveness. Importantly, two interpretations could follow and should be further investigated. One is that the effectiveness of XAI is highly conditional on contextual factors, requiring more precise, theory-driven accounts of when and for whom specific explanation types work. The other is that the field may have overestimated the psychological distinctiveness of different XAI paradigms, while underestimating the role of contextual fit in shaping how explanations are perceived and used.

Finally, our review highlights several broader methodological challenges in the current XAI literature. As expected for a relatively young and rapidly growing field, many studies do not yet meet high standards of methodological rigor, especially for user studies. Many studies vary multiple explanation aspects simultaneously, rely on diverse and sometimes weak, non-validated outcome measures, provide limited or incomplete statistical reporting, or rely on relatively small samples. These issues do not invalidate existing findings, but they make it more difficult to compare results across studies and identify the mechanisms underlying explanation effectiveness. As the field matures, greater methodological rigor, standardized outcome measures, and more transparent reporting practices will be important for building a cumulative evidence base.

6.2. Limitations of this Study

The findings of this review should be interpreted within the scope of the available evidence and the methodological choices made in this study. First, although we included a large number of studies and selected those with direct comparisons between explanation types, our conclusions remain both dependent on the quality of the underlying literature and limited in isolating causal factors. Many studies still vary multiple aspects of explanation design simultaneously, which makes it difficult to attribute observed effects to a single component. As a result, the patterns identified should be interpreted as broad tendencies rather than precise causal effects, and reflect both the strengths and limitations of the available empirical evidence.

A limitation of this comparative approach is that it does not allow conclusions about whether the explanation conditions themselves outperform the absence of explanations. Consequently, findings should be interpreted as differences between explanation designs rather than as evidence for the overall effectiveness of XAI.

In addition, the distinction between method, content, format, and interaction should be interpreted as an analytical framework for this review rather than categories that were consistently defined or reported in the original studies. Although these four aspects provided a useful structure for synthesizing a highly heterogeneous literature, many studies did not explicitly describe their comparisons in these terms. Consequently, the classification of explanation characteristics required interpretation by the reviewers based on the reported explanation designs and materials. While we applied this framework consistently across studies, some explanation implementations could reasonably be classified in multiple ways, which may have introduced ambiguity in how individual comparisons were categorized.

Third, the included studies show substantial heterogeneity in domains, tasks, users, and outcome measures. While this diversity provides a comprehensive view of XAI applications, it limits direct comparability and reduces the extent to which findings can be generalized beyond broad patterns.

Finally, we applied quality criteria to decide which studies to include. This improves the interpretability and credibility of our findings, as it ensures that conclusions are based on studies with sufficient methodological detail and rigor. At the same time, this decision may have influenced the patterns we observe, as excluding lower-quality studies also means excluding part of the available evidence. The exact threshold is inevitably somewhat arbitrary, but it was necessary to strike a balance between including as much literature as possible and ensuring that the results could be meaningfully compared and interpreted.

6.3. Conclusion

In this paper, we conducted a systematic review of studies that directly compare different implementations of explainable AI (XAI), with the aim of understanding how explanation design influences human–AI interaction outcomes. Across the reviewed literature, we find that differences between explanation approaches are often limited and highly context-dependent. Comparisons between explanation methods rarely yield consistent advantages, suggesting that technical differences in how explanations are generated are often less decisive for user outcomes. In contrast, aspects that are more closely tied to the human side, such as how explanations are presented, structured, and how interaction is facilitated, more frequently influence users’ understanding, experience, and decision-making.

These findings suggest that the effectiveness of XAI lies less in the underlying algorithmic approach and more in how explanations are designed, presented, and integrated into user interaction. In other words, explanations appear to become more impactful as they move closer to the human-centered end of the spectrum of explanation aspects.

Taken together, our results highlight the need to shift the focus of XAI research from developing new explanation methods toward designing explanations that are aligned with human needs, contexts, and decision-making processes. Rather than asking which explanation method is best, future research should focus on when, for whom, and in which context explanations truly make a difference.