"How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability

1. Introduction

Artificial Intelligence (AI) systems constitute multiple components, such as the data, models, predictions and the users who will use such systems. Explainable AI (XAI) methods are designed to elucidate these different components of “black-box” AI systems [1,2]. The act of offering explanations throughout the distinct components of an AI system is also referred to as dimensions of explainability [3,4]. Recent works have demonstrated the benefits of combining different types of XAI methods in multifaceted explanations, particularly for non-expert users in AI [5,6,7].

Additionally, it has been argued that the No Free Lunch theorem in Machine Learning [8] (i.e., there is no one-size fits all algorithm for all tasks or datasets) also applies to XAI, as specific XAI methods can only explain certain specific dimensions of explainability [7]. Thus, it is essential to include multiple types of XAI methods for a holistic explanation of AI systems. For example, an XAI system with both feature importance explanations and counterfactual explanations can provide more holistic explanations rather than a system using only one of these methods. However, these methods elucidate completely different dimensions of explainability. The former tries to explain the importance of certain factors considered by the model for generating predictions (i.e., model-centric explanations), whereas the latter provides various conditions for obtaining a different prediction (i.e., outcome-centric explanations). Therefore, a specific XAI method can elucidate a specific dimension of explainability.

The process of user-centric evaluation for XAI predominantly involves user-reported scores for metrics such as understandability, trust, actionability, stability, usefulness, and etc., from the participants involved in user studies [9,10,11,12]. Despite the significance of such methods, their implications can be limited for evaluating an XAI system as they do not consider different explainability dimensions other than the user perspective. Moreover, it could be difficult to compare diverse XAI methods only based on user perspectives due to certain limitations, such as users agreeing to misleading explanations even when the system makes incorrect predictions [13,14,15].

To overcome these challenges, researchers have also adopted other approaches that provide an objective evaluation of XAI methods, such as task-driven approaches [16,17], algorithmic evaluation metrics [18], and model-specific measurement metrics similar to Quantus [19]. However, estimating the individual impact of a particular XAI method becomes challenging in the presence of multiple combined XAI methods within a system. Thus, despite various objective and subjective metrics for measuring XAI methods [20], comparing diverse methods considering the different dimensions of explainability remains an unsolved problem [5]. To enable comparative evaluation studies, there is an evident need for a normalised XAI evaluation approach that considers both subjective and objective metrics across the various dimensions of explainability. This necessity raises an essential open research question: “How can we compare XAI methods elucidating different dimensions of explainability when used in XAI systems?”

In this position paper, we discuss the necessity of a standardised approach for evaluating diverse explainability methods used within XAI systems, considering the diverse explainability dimensions. We also discuss the potential advantages of adopting a standardised approach, which would enable comprehensive evaluations of explainability across systems. We believe that existence of such an approach can be extremely beneficial for the Human Centred XAI (HCXAI) community for evaluating XAI systems.

2. The Needs for a Standardised Evaluation Approach for Explainable AI Systems

A standardised evaluation methodology for assessing the explainability of diverse XAI methods within AI systems could effectively mitigate challenges associated with cross-dimensional comparisons. Such an approach should integrate both objective and subjective evaluation metrics, standardised to a consistent scale. By normalising evaluation scores across dimensions, such an approach could facilitate meaningful comparisons of the efficacy of various XAI methods in elucidating the multifaceted components of AI systems.

We highlight the following needs that could be fulfilled by a standardised evaluation approach for XAI systems:

• Holistic Measurement: This metric should provide a standardised and holistic measure of the effectiveness of different XAI methods in elucidating the multiple components of AI systems. This addresses the need for a comprehensive evaluation that goes beyond individual metrics.
• Flexibility: The standardised approach should offer flexibility in incorporating existing evaluation metrics across individual dimensions of explainability. This should encompass objective measures assessing training data quality, model performance, and considerations for prediction uncertainty alongside user-centric evaluations such as trustworthiness, understandability, and others.
• Model-Agnostic Property: This normalised evaluation approach should be model-agnostic, i.e., it could be applied to evaluate any XAI method or AI models. A model-agnostic evaluation approach could broaden its applicability to different application domains and diverse AI systems.
• Intra-System and Inter-System Comparison: The normalised approach should be used for comparing two or more XAI systems. It should also allow individual XAI methods across the different explainability dimensions. However, the main goal of such an approach is to compare different XAI methods within a system, using both subjective and objective measures.

3. Summary

To summarise, in this position paper, we discuss the needs for an approach designed to assess different dimensions of explainability of diverse XAI methods within XAI systems. We also discuss the benefits and opportunities for leveraging this approach for evaluating XAI systems. We eagerly seek feedback from the workshop participants to refine and operationalise these ideas into a robust evaluation framework.