Pre-hoc and Co-hoc Explainability: Novel Frameworks for Integrating Interpretability into Machine Learning Training for Enhanced Transparency and Performance

1. Introduction

Machine learning models are increasingly used to support decision-making in various fields, from personalized medical diagnosis to credit risk assessment and criminal justice. However, the increasing reliance on powerful black-box models raises concerns about their transparency, interpretability, and trustworthiness [1,2,3]. Understanding why a model made a particular prediction is crucial for auditing models, detecting potential biases and errors, and supporting model accountability and fairness. Explainable Artificial Intelligence (XAI) has emerged as a new research area that focuses on machine learning interpretability. The goal is to build interpretable models that will generate high-performing machine learning predictions [4] and thus enable human users to understand the models and trust them. In machine learning, the term explainability still lacks a common meaning, and the capability varies from application to application. Interpretability is often used instead. However, traditionally, explainability or interpretability refers to the ability of an artificial intelligence system to be understood by humans [5].

Explainable AI helps build trust in machine learning systems by providing insights into how models make decisions. This is particularly important in high-stakes domains such as healthcare, finance, and criminal justice, where the consequences of incorrect or biased decisions can be severe. When users understand how a model arrives at a particular output, they are more likely to trust and rely on the system [6,7]. Explanations can help identify errors, biases, and unexpected behaviors in machine learning models. Developers can debug and improve their models by understanding how features influence predictions, leading to more accurate and reliable systems [8,9]. In some domains, there are legal and regulatory requirements to explain algorithmic decisions. For example, the European Union’s General Data Protection Regulation (GDPR) includes a “right to explanation" for individuals subject to automated decision-making. Explainable AI techniques can help organizations comply with these regulations [10,11]. Explainable AI can also help uncover biases and unfairness in machine learning models [12,13]. Furthermore, explainable AI enables effective human-AI collaboration by providing a common understanding between humans and machines [14,15].

Several approaches have been proposed to explain black-box models, ranging from local methods that provide explanations for individual predictions to global methods that aim to capture the model’s overall behavior. Post-hoc explanations, such as LIME (Local Interpretable Model-Agnostic Explanations) [16], SHAP (Shapley Additive Explanations) [17], and Grad-CAM (Gradient Weighted Class Activation Mapping) [18], have gained popularity in recent years as a way to explain black-box models by perturbing the input data and learning a surrogate model that approximates the original model’s behavior locally. Although these methods can effectively generate explanations, they have been criticized for several reasons. First, the explanations may not reflect the true mechanisms of the original model but rather a simplified version that is easier to interpret [19]. Second, the surrogate model may not be faithful to the behavior of the original model in some cases, leading to potentially misleading explanations and being open to adversarial attacks [20]. Third, the perturbation of input data can alter the features’ semantics, rendering the explanations invalid or misleading and creating unstable explanations that arise with models already trained [21,22].

To address these limitations, some researchers have proposed using inherently interpretable models, such as decision trees or linear models, instead of black-box models for high-stakes decision-making [23]. However, this approach may come at the cost of reduced prediction accuracy, as interpretable models may not be able to capture the complexity of some datasets as well as black-box models. Moreover, this approach cannot be applied to models that are already deployed and running. Replacing existing black-box models in production with interpretable models requires re-training the whole model from scratch, which can be resource-intensive and time-consuming.

We propose two novel approaches to improving the explainability of black-box models, which we call pre-hoc explainability and co-hoc explainability. Our approach aims to incorporate explanations derived from an inherently interpretable white-box model into the original model’s learning stage without compromising its high prediction accuracy. Instead of learning a post-hoc white-box model, our idea is to learn a white-box model that is explainable from the start and then let this explainer model guide the learning of the black-box predictor model. This approach aims to address the limitations of post-hoc explanations, such as potential discrepancies between the explainer and the black-box model [24], and the computational overhead associated with generating explanations after model training [20].

To accomplish this goal, we design two different frameworks: (1) A Pre-Hoc Explainable Predictive Framework, where the white-box model regularizes the black-box model for optimized fidelity, and (2) A Co-hoc Explainable Predictive Framework, where the white-box and black-box models are optimized simultaneously with a shared loss function that enforces fidelity.

In the Pre-Hoc Explainable Predictive Framework, we first train the explainer model g and then use it to guide the learning of the predictor model f. The objective function for training the predictor model includes a fidelity term that minimizes the distance between the predictor’s and explainer’s outputs, encouraging the predictor to mimic the explainer’s behavior. This approach ensures that the predictor model is regularized by the explainer model, leading to improved interpretability.

In the Co-hoc Explainability Framework, we jointly optimize the predictor model f and the explainer model g during training. The shared loss function consists of both models’ standard supervised learning objective (e.g., cross-entropy loss) and a fidelity term that minimizes the distance between their outputs. By simultaneously training both models, we encourage the predictor to learn from the explainer and the explainer to adapt to the predictor, resulting in a more coherent and interpretable system.

Our proposed frameworks differ from existing approaches in several aspects. First, we integrate interpretability directly into the model training process rather than relying on post-hoc explanations. Second, we use a transparent white-box model to guide the learning of the black-box model, ensuring that the explanations are faithful to the predictor’s behavior. Finally, our frameworks are model-agnostic and can be applied to any differentiable predictor and explainer models.

The main contributions of this paper are:

• We propose two novel approaches to improving the explainability of black-box models, called pre-hoc explainability and co-hoc explainability, which leverage the insights provided by an inherently interpretable white-box model to guide the training of the black-box model in a way that preserves its accuracy while enhancing its global interpretability. These approaches integrate explainability directly into the training process, ensuring that the explanations are faithful to the model’s behavior and do not require additional post-hoc computations.
• We extend our pre-hoc explainability framework to provide local explanations by incorporating the Jensen-Shannon (JS) divergence [25] as a regularization term in the loss function. This allows our method to generate instance-specific explanations that capture the local behavior of the black-box model, similar to post-hoc methods such as LIME [16]. However, unlike LIME, our approach integrates local explainability directly into the training process, ensuring that the explanations are faithful to the model’s behavior and consistent with the model’s predictions for similar instances.
• Unlike post-hoc explanations, our approaches do not rely on random input perturbation and post-secondary model learning, thus avoiding the potential pitfalls of surrogate modeling, such as instability and unfaithfulness [20,26]. This makes them more scalable, robust, and reliable in practice. By incorporating global and local explainability through an interpretable white-box model and the JS divergence, our methods can generate more accurate and stable explanations compared to LIME.
• We enhance the accuracy of white-box models through the co-hoc learning framework. The white-box model, which is learned for the purpose of explaining the black-box predictor, achieves significantly higher prediction accuracy after the co-hoc learning process. This finding highlights the potential of the co-hoc in-training approach to improve the performance of white-box models, which are essential and required in certain high-risk and regulated application tasks in healthcare and legal decision-making.
• We demonstrate the effectiveness of our approaches on three benchmark datasets from diverse domains (credit risk scoring and movie recommendations), showing that they outperform traditional black-box models in terms of fidelity while maintaining comparable accuracy. We also compare our methods with the LIME post-hoc explainability technique [16] in terms of the quality and stability of the generated explanations, as well as the computational cost associated with generating explanations.

The remainder of this paper is organized as follows: Section 2 reviews related work in explainable artificial intelligence. Section 3 presents our proposed pre-hoc and co-hoc explainability frameworks and their theoretical foundation and describes our experimental setup and evaluation metrics. Section 4 presents and discusses the results of our experiments. Finally, Section 5 concludes the paper and outlines potential future research directions.

2. Related Work

2.1. Explainability Approaches in Machine Learning

The landscape of explainable artificial intelligence (XAI) encompasses several distinct approaches. Inherently interpretable models, such as linear models [27,28] and decision trees [29,30], offer transparent decision-making processes at the potential cost of reduced predictive power. While these white-box models provide clarity, they often cannot match the performance of complex black-box models in challenging tasks [23].

Post-hoc explainability techniques aim to explain already-trained black-box models. LIME (Local Interpretable Model-Agnostic Explanations) [16] approximates complex models locally around specific predictions using simpler, interpretable surrogate models. LIME solves the optimization problem:

$$e(x,f):=argminF\left(f,g,N_x\right)+\Omega \left(g\right)$$

(1)

where $\Omega \left(g\right)$ represents a regularizer that encourages desirable properties like sparsity. Similarly, SHAP (SHapley Additive exPlanations) [17] applies game theory concepts to assign feature importance values. Despite their popularity, these approaches have been criticized for generating potentially misleading or unstable explanations [20,26] and introducing computational overhead during inference [19].

Model-specific explainability techniques, or explainability by design, involve architectural adjustments to improve model interpretability. These approaches modify model architecture to enhance comprehensibility [31,32,33,34] and often employ regularization techniques to promote sparsity and interpretability [35,36].

2.2. In-Training Explainability Techniques

While most XAI research has focused on post-hoc explainability or inherently interpretable models, in-training explainability represents a less explored but promising direction. Tree regularization [37] has been used to train deep time-series models with a focus on human-simulability [38]. Others have proposed training models with latent explainability but still rely on post-hoc explanations [39,40].

Alternative approaches include game-theoretic methods between predictor and explainer [41,42], using cooperative games to optimize explainers for locality. The EXPO framework [43] applied regularization to push black-box models toward interpretable features, but their explanations remain post-hoc, specifically optimized for LIME’s neighborhood-based fidelity, which requires computation at prediction time. Concept-based explanation methods [44] learn latent explanations during training but are limited to special input types or domains with available external supervision.

Current research highlights the need for optimization during training and model-agnostic methods to improve global explainability. Our approach addresses this gap by directly incorporating global interpretability into black-box learning at training time through an interpretable explainer model that doesn’t require additional post-hoc computation at prediction time.

2.3. Explanation Types and Evaluation

XAI explanations can be categorized into several types, each offering different perspectives and serving distinct purposes. Rule-based explanations transform model decisions into human-readable rules, making them particularly valuable in domains requiring transparency, such as healthcare and finance [45,46]. Decision trees exemplify this approach by breaking decisions into comprehensible if-then rules. While intuitive, these explanations may become unwieldy as model complexity increases.

Feature-based explanations quantify the contribution of individual features to predictions [16,47]. LIME and SHAP are prominent techniques in this category, with LIME approximating local behavior using interpretable models and SHAP assigning feature importance based on game theory principles. These methods provide valuable insights into which inputs most strongly influence outputs, enabling feature selection and bias detection. However, they may not capture complex feature interactions or non-linear relationships effectively [48].

Concept-based explanations bridge the gap between model internals and human understanding by expressing decisions through high-level concepts [49,50]. Techniques like Concept Activation Vectors (CAVs) align model behavior with human-understandable concepts, which is particularly useful in domains with established conceptual frameworks like medicine. These explanations facilitate communication with domain experts but require well-defined concepts and may not capture the model’s full complexity.

Instance-based explanations identify specific training examples that significantly influence predictions [8,51]. Techniques like influence functions measure how individual training instances affect model outputs, while prototype selection identifies representative examples that characterize decision boundaries. These concrete explanations help users understand model behavior through familiar examples, though interpretation can be challenging when influential instances aren’t intuitively related to predictions.

Local explanations focus on individual predictions, explaining specific decisions rather than overall model behavior [16,47]. These targeted insights are crucial in domains where individual decisions carry significant consequences. Conversely, global explanations describe the model’s general behavior across all instances [45,52], providing a holistic understanding of the model’s decision logic through interpretable surrogate models or feature importance methods.

Multi-model explanations generate comparative views across different models [53,54], helping users understand similarities and differences in how various models process information. This comparative approach provides insights into model robustness and generalizability, but can be computationally expensive and challenging to reconcile when models yield conflicting explanations.

Selecting appropriate explanation types depends on factors including the target audience, model complexity, and application domain. In healthcare, concept-based explanations may better communicate with medical professionals, while rule-based explanations might be more suitable for patient-facing applications [55]. Different explanation types can be combined for comprehensive understanding—for example, aggregating local explanations to generate global insights [56].

Post-hoc explainability can be evaluated using three key metrics: point fidelity, neighborhood fidelity [43], and stability [26]. Point fidelity measures agreement between the explainer and predictor for individual instances:

$$\mathrm{PointFidelity}\left({\mathbf{x}}_i\right)=\mathbb{1}({\widehat{y}}_{f,i}={\widehat{y}}_{lime,i})$$

(2)

The average point fidelity across all instances is calculated as:

$$\mathrm{AvgPointFidelity}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathrm{PointFidelity}\left({\mathbf{x}}_i\right)$$

(3)

Neighborhood fidelity extends this concept to consider agreement within local neighborhoods around each instance, providing a more robust measure of explanation quality:

$$\mathrm{NeighborhoodFidelity}\left({\mathbf{x}}_i\right)={\displaystyle \frac{1}{k}}\sum _{{\mathbf{x}}_j\in {\mathcal{N}}_k\left({\mathbf{x}}_i\right)}\mathbb{1}({\widehat{y}}_{f,j}={\widehat{y}}_{lime,j})$$

(4)

where ${\mathcal{N}}_k\left({\mathbf{x}}_i\right)$ denotes the set of k-nearest neighbors of instance ${\mathbf{x}}_i$ in the feature space. This metric assesses how well explanations generalize to similar instances, capturing the local coherence of explanations.

The stability metric quantifies the variability in fidelity scores using total variation:

$$\mathrm{TV}\left(\mathcal{F}\right)={\displaystyle \frac{1}{N-1}}\sum _{i=1}^{N-1}|f_{i+1}-f_i|$$

(5)

where $\mathcal{F}=\{f_1,f_2,\dots ,f_N\}$ represents a set of fidelity scores. Lower total variation indicates higher stability, suggesting that explanations remain consistent across different instances without sudden fluctuations. Stability is crucial for building trust in explanations, as users expect similar explanations for similar instances.

These evaluation metrics provide complementary perspectives on explanation quality. Point fidelity focuses on individual accuracy, neighborhood fidelity considers local coherence, and stability measures consistency across instances. A desirable explainable model would achieve high average fidelity scores and low total variation, indicating explanations that are both accurate and stable across different instances. Together, these metrics offer a comprehensive framework for assessing explanation quality that aligns with the multifaceted nature of interpretability [5,57].

2.4. Factorization Machines

Factorization Machines (FMs) [58] are supervised learning models applicable to various prediction tasks including regression, classification, and ranking. The model equation for a degree-2 FM is:

$$\widehat{y}\left(x\right)=w_0+\sum _{i=1}^n{w}_i{x}_i+\sum _{i=1}^n\sum _{j=i+1}^n\langle {\mathbf{v}}_i,{\mathbf{v}}_j\rangle x_i{x}_j$$

(6)

Despite their effectiveness, FMs lack transparency. The model parameters (${\mathbf{w}}_{\mathbf{0}},\mathbf{w},\mathbf{V}$) include latent factors that make interpretability challenging. Recent efforts to improve FM transparency include Subspace Encoding Factorization Machines (SEFM) [59], Knowledge-aware Hybrid Factorization Machines (kaHFM) [60], and Attentional Factorization Machines (AFM) [61].

Our proposed pre-hoc and co-hoc explainability frameworks address the limitations of both post-hoc techniques and inherently interpretable models by integrating explainability directly into the training process of black-box models, ensuring faithful explanations without compromising accuracy.

3. Methodology

3.1. Problem Formulation

Let $\mathcal{S}={\left\{({\mathbf{x}}_i,{\mathbf{y}}_i)\right\}}_{i=1}^{\mathcal{N}}\subset \mathcal{Z}$ be a sample from a distribution $\mathcal{D}$ in a domain $\mathcal{Z}=\mathcal{X}\times \mathcal{Y}$, where $\mathcal{X}$ is the instance space and $\mathcal{Y}$ is the label space. We learn a differentiable predictive function $f\in \mathcal{F}:\mathcal{X}\to \mathcal{Y}$ together with a transparent explainer function $g\in \mathcal{G}:\mathcal{X}\to \mathcal{Y}$ defined over a functional class $\mathcal{G}$. We refer to functions f and g as the predictor and the explainer, respectively, throughout the paper. $\mathcal{G}$ is strictly constrained to be an inherently explainable functional set, such as a set of linear functions or decision trees. We assume that we have a distance function $d:\mathcal{X}\times \mathcal{Y}\to {\mathbb{R}}_{\ge 0}$ such that $d(y,\widehat{y})=0⟷y=\widehat{y}$, which measures the point-wise similarity between two probability distributions in $\mathcal{Y}$ and can be used to optimize f and g.

Instead of learning a post-hoc white-box model, our idea is to learn a white-box model that is explainable from the start and then let this explainer model guide the learning of the black-box predictor model. To accomplish this goal, we design two different frameworks: (1) A Pre-Hoc Explainable Predictive Framework, where the white-box model regularizes the black-box model for optimized fidelity, and (2) A Co-hoc Explainable Predictive Framework, where the white-box and black-box models are optimized simultaneously with a shared loss function that enforces fidelity.

3.1.1. Enforcing Fidelity

Given an inherently interpretable white-box model g with parameters $\varphi$, let its predictions result in a probability distribution $p_{\varphi }$. Given the black-box model f with parameters $\theta$, let its predictions result in probability distribution $p_{\theta }$ over K classes $y\in \mathcal{Y}=\{1,2,..,K\}$. We propose a fidelity objective function, which measures the point-wise probability distance between $p_{\varphi }$ and $p_{\theta }$, which are respectively the outputs of g and f for all given input data $\mathcal{X}$. The optimization problem is formulated as:

$$\underset{f\in \mathcal{F}}{min}{\displaystyle \frac{1}{N}}\sum _{i=1}^{N}D\left(f\left(x_i\right),g\left(x_i\right)\right),$$

(7)

where function D is a divergence distance measurement, such as the Jensen-Shannon divergence [62]. We use $D_{JS}$, Jensen-Shannon divergence, to measure the point-wise deviation of the predictive distributions $f_{\theta }$ and $g_{\varphi }$.

Denote by $\mathcal{P}$ the set of probability distributions. The Kullback-Leibler divergence (KL). KL : $\mathcal{P}\times \mathcal{P}\to [0,\infty ]$ is a fundamental distance between probability distributions in $\mathcal{D}$ [63], defined by:

$$D_{\mathrm{KL}}\left(p\right|\left|q\right):=\int plog{\displaystyle \frac{p}{q}}\mathrm{d}\mu ,$$

(8)

where p and q denote probability measures P and Q with respect to $\mu$.

Let $p,q\in {\Delta }^{K-1}$ have the corresponding weights $\pi ={\left[{\pi }_1,{\pi }_2\right]}^T\in \Delta$. Then, the Jensen-Shannon divergence between p and q is given by:

$$\begin{array}{cc}\hfill D_{\mathrm{JS}}\left(p,q\right)& :=H\left(m\right)-{\pi }_{1}H\left(p\right)-{\pi }_{2}H\left(q\right)\hfill \\ \hfill & ={\pi }_1{D}_{\mathrm{KL}}\left(p\parallel m\right)+{\pi }_2{D}_{\mathrm{KL}}\left(q\parallel m\right),\hfill \end{array}$$

(9)

with H the Shannon entropy, and $\mathbf{m}={\pi }_{1}p+{\pi }_{2}q$. Unlike the Kullback-Leibler divergence $\left(D_{\mathrm{KL}}\left(p\parallel q\right)\right)$, JS is symmetric, bounded, and does not require absolute continuity.

We propose a fidelity objective function, ${\mathcal{L}}_{JS}$, that is calculated using the Jensen-Shannon divergence (JS), as follows:

$${\mathcal{L}}_{\mathrm{JS}}\left(x_{1:N},f_{\theta },g_{\varphi }\right):=D_{\mathrm{JS}}({\widehat{y}}_{\varphi },{\widehat{y}}_{\theta })$$

(10)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{JS}}\left(x_{1:N},f_{\theta },g_{\varphi }\right)& :={\displaystyle \frac{1}{2}}(D_{\mathrm{KL}}({\widehat{y}}_{\varphi }\Vert {\displaystyle \frac{({\widehat{y}}_{\varphi }+{\widehat{y}}_{\theta })}{2}})\hfill \\ \hfill & +D_{\mathrm{KL}}({\widehat{y}}_{\theta }\Vert {\displaystyle \frac{({\widehat{y}}_{\varphi }+{\widehat{y}}_{\theta })}{2}}))\hfill \end{array}$$

(11)

Our proposed fidelity objective function has three distinct regularization properties:

Bounded Regularizer: The Jensen–Shannon divergence distance is always bounded:

$$0\le D_{\mathrm{JS}}(p:q)\le \sqrt{log2}$$

(12)

Symmetry Preserving Regularizer: JS is symmetry preserving if the corresponding weights $\pi =\left[{\pi }_1,{\pi }_2\right]$ are selected as $\pi =\left[{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right]$.

Differentiable Regularizer: The regularizer is differentiable, which means that it can be easily incorporated into the training process using standard gradient descent update rules and backpropagation techniques.

3.2. Pre-hoc Explainability Framework

The Pre-hoc Explainability Framework uses a modified learning objective to incorporate explanations during the training process. In this framework, we first train a white-box explainer model $g_{\varphi }$ on the training data. Then, we use this explainer model to guide the learning of the black-box predictor model $f_{\theta }$ by including a fidelity term in the loss function.

The loss function for the Pre-hoc framework is formulated as:

$${\mathcal{L}}_{\mathrm{Pre}-\mathrm{hoc}}={\mathcal{L}}_{\mathrm{BCE}}+{\lambda }_1{D}_{\mathrm{JS}}+{\lambda }_2{\mathcal{L}}_2,$$

(13)

where ${\mathcal{L}}_{\mathrm{BCE}}$ is the binary cross-entropy loss that ensures accurate predictions, ${\lambda }_1$ is an explainability regularization coefficient that controls the trade-off between explainability and accuracy, and ${\lambda }_2$ is the coefficient for standard ${\mathcal{L}}_2$ regularization of model parameters $\theta$ that aims to avoid overfitting and exploding gradients.

Expanding the loss function, we get:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{\mathrm{Pre}-\mathrm{hoc}}(\theta ,\varphi ,X,y)=\underset{\mathrm{Predictor}\mathrm{Accuracy}}{\underbrace{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N-y_{i}log\left({\widehat{y}}_{\theta ,i}\right)-(1-y_i)log\left(1-{\widehat{y}}_{\theta ,i}\right)}}\hfill \\ \hfill & +\underset{\mathrm{Fidelity}}{\underbrace{{\lambda }_1{\displaystyle \frac{1}{2}}({\sum }_{i=1}^{N}ln\left({\displaystyle \frac{{\widehat{y}}_{\varphi ,i}}{{\widehat{y}}_{\theta ,i}}}\right){\widehat{y}}_{\varphi ,i}+{\sum }_{i=1}^{N}ln\left({\displaystyle \frac{{\widehat{y}}_{\theta ,i}}{{\widehat{y}}_{\varphi ,i}}}\right){\widehat{y}}_{\theta ,i})}}+\underset{\mathrm{L}2\mathrm{Regularization}}{\underbrace{{\lambda }_2{\sum }_j{\theta }_j^2}},\hfill \end{array}$$

(14)

Since the explanation is provided by the white-box model $g_{\varphi }$ that is inherently interpretable, the transparency is considered high when the explainer model outputs ${\widehat{y}}_{\varphi }$ are similar to the regularized model $f_{\theta }$ outputs ${\widehat{y}}_{\theta }$. This similarity is captured by $D_{\mathrm{JS}}$, which is the Fidelity term in the objective function ${\mathcal{L}}_{\mathrm{Pre}-\mathrm{hoc}}$ (Eq. 13). While the objective function is to learn a model that will make accurate predictions, we give greater importance to model predictions that are similar to the white-box predictions and penalize those that are not similar.

The optimization procedure for the Pre-hoc framework is as follows:

• Train the white-box explainer model $g_{\varphi }$ on the training data to minimize the binary cross-entropy loss.
• Fix the parameters $\varphi$ of the explainer model.
• Train the black-box predictor model $f_{\theta }$ to minimize the combined loss function ${\mathcal{L}}_{\mathrm{Pre}-\mathrm{hoc}}$.

3.3. Co-Hoc Explainability Framework

In contrast to the Pre-hoc framework, the Co-hoc Explainability Framework jointly optimizes both the predictor model $f_{\theta }$ and the explainer model $g_{\varphi }$ during training. This approach allows the explainer to adapt to the predictor and vice versa, resulting in a more coherent and interpretable system.

The loss function for the Co-hoc framework is given by:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{\mathrm{Co}-\mathrm{hoc}}(\theta ,\varphi ,X,y)=\underset{\mathrm{Predictor}\mathrm{Accuracy}}{\underbrace{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N-y_{i}log\left({\widehat{y}}_{\theta ,i}\right)-(1-y_i)log\left(1-{\widehat{y}}_{\theta ,i}\right)}}\hfill \\ \hfill & +\underset{\mathrm{Explainer}\mathrm{Accuracy}}{\underbrace{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N-y_{i}log\left({\widehat{y}}_{\varphi ,i}\right)-(1-y_i)log\left(1-{\widehat{y}}_{\varphi ,i}\right)}}\hfill \\ \hfill & +\underset{\mathrm{Fidelity}}{\underbrace{{\lambda }_1{\displaystyle \frac{1}{2}}({\sum }_{i=1}^{N}ln\left({\displaystyle \frac{{\widehat{y}}_{\varphi ,i}}{{\widehat{y}}_{\theta ,i}}}\right){\widehat{y}}_{\varphi ,i}+{\sum }_{i=1}^{N}ln\left({\displaystyle \frac{{\widehat{y}}_{\theta ,i}}{{\widehat{y}}_{\varphi ,i}}}\right){\widehat{y}}_{\theta ,i})}}\hfill \\ \hfill & +\underset{\mathrm{Predictor}\mathrm{Regularization}}{\underbrace{{\lambda }_2{\sum }_j{\theta }_j^2}}+\underset{\mathrm{Explainer}\mathrm{Regularization}}{\underbrace{{\lambda }_3{\sum }_j{\varphi }_j^2}},\hfill \end{array}$$

(15)

The primary distinction between the Co-hoc and Pre-hoc frameworks lies in the joint optimization of the predictor $f_{\theta }$ and explainer $g_{\varphi }$ through simultaneous stochastic gradient descent with mini-batches. In the Co-hoc framework, both models are trained to minimize the combined loss function ${\mathcal{L}}_{\mathrm{Co}-\mathrm{hoc}}$, which includes accuracy terms for both models, a fidelity term, and regularization terms.

The optimization procedure for the Co-hoc framework is as follows:

(1)

Initialize the parameters $\theta$ of the predictor model and $\varphi$ of the explainer model.

(2)

For each mini-batch of training data:

(a)

Compute the predictions of both models: ${\widehat{y}}_{\theta }$ and ${\widehat{y}}_{\varphi }$.

(b)

Calculate the combined loss function ${\mathcal{L}}_{\mathrm{Co}-\mathrm{hoc}}$.

(c)

Update both sets of parameters $\theta$ and $\varphi$ using gradient descent.

3.4. Generating Explanations

Once the models are trained, the white-box explainer model $g_{\varphi }$ naturally provides interpretable explanations for the predictions made by the black-box model $f_{\theta }$. For linear models, the feature importance scores can be directly derived from the model coefficients, providing a global understanding of feature relevance across the entire dataset.

For a specific instance, we generate feature importance scores based on the trained white-box model and the feature values. The importance score of feature j for instance i is calculated as:

$$\mathrm{Importance}(j,i)={\varphi }_j·{\mathrm{MAD}}_j$$

(16)

where ${\varphi }_j$ is the coefficient of feature j in the trained white-box model, and ${\mathrm{MAD}}_j$ is the mean absolute deviation of feature j across the dataset, calculated as:

$${\mathrm{MAD}}_j={\displaystyle \frac{1}{N}}\sum _{i=1}^N|x_{ij}-{\overline{x}}_j|$$

(17)

where $x_{ij}$ is the value of feature j for instance i, and ${\overline{x}}_j$ is the mean value of feature j across all instances.

The mean absolute deviation serves as a scaling factor to normalize feature importance scores, ensuring that the scores are comparable across different features and datasets. By incorporating the MAD in the importance score calculation, we account for the variability and scale of the features, providing a more reliable and interpretable measure of feature importance.

3.5. Extending to Local Explainability

While the frameworks described above provide global explanations that capture the overall behavior of the black-box model, they may not adequately explain individual predictions. To address this limitation, we extend our pre-hoc and co-hoc frameworks to incorporate local explainability, enabling the generation of instance-specific explanations.

3.5.1. Local Explainability with Neighborhood Information

Local explainability refers to understanding and interpreting a model’s predictions at an individual instance level. We leverage neighborhood information and the Jensen-Shannon divergence to achieve local explainability. By considering each instance’s local neighborhood and comparing the predictions of the black-box model with those of the white-box model within this neighborhood, we can capture the regional variations in predictions and ensure that the explanations are faithful to the model’s local behavior.

We define the neighborhood fidelity objective function as follows:

$$\underset{f_{\theta }\in \mathcal{F}}{min}{\displaystyle \frac{1}{N}}\sum _{i=1}^N{D}_{local}\left(f_{\theta }\left(\mathcal{N}\left(x_i\right)\right),g_{\varphi }\left(\mathcal{N}\left(x_i\right)\right)\right),$$

(18)

where $\mathcal{N}\left(x_i\right)$ denotes the set of instances in the local neighborhood of instance $x_i$, and $D_{local}$ is a divergence measure such as the Jensen-Shannon divergence computed over the neighborhood.

The Jensen-Shannon divergence for local explainability is given by:

$$\begin{array}{cc}\hfill D_{{\mathrm{JS}}_{\mathrm{local}}}({\widehat{y}}_{\varphi },{\widehat{y}}_{\theta })& ={\displaystyle \frac{1}{2}}(\sum _{x_j\in \mathcal{N}\left(x_i\right)}ln\left({\displaystyle \frac{{\widehat{y_j}}_{\varphi }}{{\widehat{y_j}}_{\theta }}}\right){\widehat{y_j}}_{\varphi }\hfill \\ \hfill & +\sum _{x_j\in \mathcal{N}\left(x_i\right)}ln\left({\displaystyle \frac{{\widehat{y_j}}_{\theta }}{{\widehat{y_j}}_{\varphi }}}\right){\widehat{y_j}}_{\theta })\hfill \end{array}$$

(19)

3.5.2. Two-Phase Approach for Local Explainability

Our approach to local explainability consists of two phases: Phase 1: Co-hoc: Integrating Local Explainability with Neighbors in Training. In the first phase, we train the black-box predictor and white-box explainer models using our pre-hoc or co-hoc frameworks, incorporating the local neighborhood information. The loss functions are modified to include the local Jensen-Shannon divergence:

For the Pre-hoc Local Explainability framework:

$${\mathcal{L}}_{\mathrm{Local}-\mathrm{Pre}-\mathrm{hoc}}={\mathcal{L}}_{\mathrm{BCE}}+{\lambda }_1{D}_{J{S}_{local}}+{\lambda }_2{\mathcal{L}}_2$$

(20)

For the Co-hoc Local Explainability framework:

$${\mathcal{L}}_{\mathrm{Local}-\mathrm{Co}-\mathrm{hoc}}={\mathcal{L}}_{\mathrm{f}-\mathrm{BCE}}+{\mathcal{L}}_{g-\mathrm{BCE}}+{\lambda }_1{D}_{J{S}_{local}}+{\lambda }_2{\mathcal{L}}_2\left(f\right)+{\lambda }_2{\mathcal{L}}_2\left(g\right)$$

(21)

Phase 2: Computing Local Explanations. In the second phase, for each test instance $x_i$, we identify its nearest neighbors from the training set, forming a local in-testing neighborhood $\mathcal{N}\left(x_i\right)$. We then fine-tune the global white-box explainer model within this neighborhood to obtain a local explainer model $g_{{\varphi }_i}^{local}$. The fine-tuning is performed by minimizing:

$${\mathcal{L}}_{\mathrm{Local}-\mathrm{Explainer}}=D_{{\mathrm{JS}}_{\mathrm{local}}}^{expl}({\widehat{y}}_{{\varphi }_i}^{local},{\widehat{y}}_{\theta })$$

(22)

Algorithm 1:Testing PHASE 2: Computing Local Explanations

This fine-tuned local explainer model provides instance-specific explanations that capture the local behavior of the black-box model around the test instance. The feature importance scores are then extracted from the local explainer model, quantifying the contribution of each feature to the model’s prediction for that specific instance.

3.6. Experimental Setup

3.6.1. Datasets

We evaluate our frameworks on three publicly accessible real-world datasets:

HELOC Dataset: The FICO HELOC dataset [64] contains 10,459 anonymized records of home equity line of credit applications. The target variable predicts whether an applicant will make payments on time.

MovieLens 100k: This dataset [65] contains 100,000 movie ratings from 1,000 users on 1,700 movies. We convert the ratings into a binary classification task, with ratings $\ge 3$ considered positive (1) and ratings $<3$ considered negative (0).

MovieLens 1M: This dataset [66] contains movie ratings, has 1 million ratings based on 6000 users on 4000 movies. We convert the ratings into a binary classification task, with ratings $\ge 3$considered positive (1) and ratings $<3$ considered negative (0).

3.6.2. Evaluation Metrics

We use the following metrics to evaluate our frameworks:

Accuracy: We measure prediction accuracy using the Area Under the ROC Curve (AUC).

Fidelity: We measure how well the white-box explainer model mimics the behavior of the black-box predictor model using AUC$(f_{\theta },g_{\varphi })$.

Point Fidelity: This measures the agreement between the explainer and predictor for individual instances:

$$\mathrm{PointFidelity}\left({\mathbf{x}}_i\right)=\mathbb{1}({\widehat{y}}_{f,i}={\widehat{y}}_{g,i})$$

(23)

Neighborhood Fidelity: This extends point fidelity to consider agreement within local neighborhoods:

$$\mathrm{NeighborhoodFidelity}\left({\mathbf{x}}_i\right)={\displaystyle \frac{1}{k}}\sum _{{\mathbf{x}}_j\in {\mathcal{N}}_k\left({\mathbf{x}}_i\right)}\mathbb{1}({\widehat{y}}_{f,j}={\widehat{y}}_{g,j})$$

(24)

Stability: This measures the consistency of explanations, calculated as the total variation of fidelity scores:

$$\mathrm{TV}\left(\mathcal{F}\right)={\displaystyle \frac{1}{N-1}}\sum _{i=1}^{N-1}|f_{i+1}-f_i|$$

(25)

Computational Efficiency: We measure the computational cost of generating explanations, including training time and explanation generation time.

3.6.3. Implementation Details

We implement our frameworks using PyTorch [67]. All models are trained using the Adam optimizer [68] with a learning rate of 0.001. Each dataset is split into training, validation, and test sets in the ratio 80:10:10. We select the optimal regularization parameter ${\lambda }_1$ from the set {0.01, 0.05, 0.25, 0.5, 1} based on validation performance. For local explainability, we set the neighborhood size to k=10 during Phase 1 and k=100 during Phase 2. All experiments are repeated five times, and the results are averaged across runs.

As our black-box model, we use Factorization Machines [69], which are widely used for classification, regression, and recommendation tasks. For the white-box explainer model, we use sparse logistic regression, which is inherently interpretable and provides feature importance scores directly from its coefficients.

4. Results

In this section, we present the experimental results of our proposed pre-hoc and co-hoc explainability frameworks. We first examine the accuracy and fidelity trade-off of our global explainability approaches. Then, we analyze the local explainability performance and compare our approaches with the state-of-the-art post-hoc explainability method LIME.

4.1. Global Explainability Results

4.1.1. Accuracy and Fidelity Trade-off

Table 1 presents the accuracy (AUC) and fidelity scores of our pre-hoc and co-hoc frameworks compared to the baseline white-box (WB) and black-box (BB) models. We observe that both of our proposed frameworks achieve higher fidelity scores than the original black-box model while maintaining comparable accuracy. The co-hoc framework consistently outperforms the pre-hoc framework in terms of fidelity across all datasets, demonstrating the effectiveness of joint optimization.

The results confirm that our proposed approaches successfully maintain high accuracy while significantly improving fidelity. Notably, the co-hoc framework achieves fidelity improvements of 10.9%, 6.8%, and 10.9% on ML-100k, ML-1M, and HELOC datasets, respectively, compared to the original black-box model. This demonstrates the effectiveness of our joint optimization approach in aligning the behaviors of the black-box predictor and white-box explainer models.

4.1.2. Effect of Regularization Parameter

Figure 1 and Figure 2 illustrate the effect of the explainability regularization parameter ${\lambda }_1$ on the precision and fidelity of our frameworks. As ${\lambda }_1$ increases, we observe a consistent improvement in fidelity scores across all datasets, with only minimal impact on accuracy. This confirms that our frameworks effectively balance the trade-off between accuracy and explainability, allowing users to control this balance through the regularization parameter.

For example, the ML-100k dataset with the pre-hoc framework, as ${\lambda }_1$ increases from 0.01 to 1.0, fidelity improves from 0.8207 to 0.9410 (a 14.6% increase), while accuracy remains relatively stable (0.7840 to 0.7740). Similarly, for the HELOC dataset, fidelity improves from 0.7482 to 0.8454 (a 9.3% increase) as ${\lambda }_1$ increases, with accuracy actually improving slightly from 0.7591 to 0.7719. These results demonstrate that our frameworks can achieve high fidelity without sacrificing accuracy.

The co-hoc framework exhibits an even more favorable trade-off curve, particularly in the HELOC dataset, where both accuracy and fidelity improve simultaneously as ${\lambda }_1$ increases. This suggests that the joint optimization approach not only aligns the behavior of the two models but also enhances their complementary strengths. The plateau observed in fidelity scores at higher ${\lambda }_1$ values (above 0.5) indicates an optimal operating point beyond which additional regularization yields diminishing returns. This behavior provides practical guidance for hyperparameter selection in real-world applications, where setting ${\lambda }_1$ between 0.25 and 0.5 offers the best balance between model performance and interpretability.

4.2. Local Explainability Results

4.2.1. Comparison with LIME

Table 2 presents the comparison of our local explainability frameworks with LIME on the HELOC in terms of point fidelity, neighborhood fidelity, and stability. Both our pre-hoc and co-hoc local explainability frameworks significantly outperform LIME across all metrics.

On the HELOC dataset, our frameworks achieve neighborhood fidelity scores of 0.9587 (pre-hoc) and 0.9647 (co-hoc), significantly higher than LIME’s score of 0.6600. Similarly, on the ML-100k dataset, our frameworks achieve neighborhood fidelity scores of 0.9597 (pre-hoc) and 0.9647 (co-hoc), outperforming LIME’s score of 0.7410. The stability of our explanations, measured by total variation, is also significantly better than LIME, indicating more consistent explanations across different instances.

4.3. Effect of Regularization Parameter on Local Explainability Metrics

Table 3 and Figure 3 shows the impact of the explainability regularization parameter ${\lambda }_1$ on the point fidelity, neighborhood fidelity, and stability metrics for the Pre-hoc framework on the ML-100k dataset. The results show a clear positive relationship between the regularization strength and the quality of the explanation.

In the absence of regularization, the model achieves moderate fidelity scores (point fidelity of 0.8183 and neighborhood fidelity of 0.8050) but exhibits lower stability with a high total variation score of 0.3524. This indicates that without explainability regularization, the explanations are less consistent across different instances, even when the model achieves reasonable alignment with the explainer.

As we train with regularization parameter and gradually increase the ${\lambda }_1$, we observe significant improvements across all metrics. With a minimal regularization of ${\lambda }_1=0.01$, point fidelity improves to 0.8473 (3.5% increase), neighborhood fidelity increases to 0.8553 (6.2% increase), and stability improves dramatically with total variation decreasing to 0.1290 (63.4% reduction). At ${\lambda }_1=0.25$, point fidelity reaches 0.9740 (19.0% increase from no regularization), neighborhood fidelity increases to 0.9903 (23.0% increase), and stability improves substantially with total variation reduced to 0.0189 (94.6% reduction). This indicates that moderate regularization significantly enhances both the alignment between the predictor and explainer models and the consistency of explanations across different instances.

At higher regularization strengths (${\lambda }_1=0.5$ and ${\lambda }_1=1.0$), the metrics continue to improve but with decreasing returns. The point fidelity reaches its peak at 0.9951 with ${\lambda }_1=1.0$, representing a 21.6% improvement over the non-regularized model. Similarly, neighborhood fidelity reaches 0.9953, a 23.6% improvement. The stability metric plateaus at 0.0078 for both ${\lambda }_1=0.5$ and ${\lambda }_1=1.0$, indicating that additional regularization beyond ${\lambda }_1=0.5$ does not further improve the consistency of explanations.

These results demonstrate that incorporating explainability regularization through the Jensen-Shannon divergence significantly enhances the quality of explanations generated by the Pre-hoc framework. Even $0.01$ regularization provides substantial benefits, with optimal performance achieved at moderate to high regularization strengths (${\lambda }_1\ge 0.25$). The improvements in fidelity metrics indicate better alignment between the predictor and explainer models, while the reduction in total variation demonstrates more consistent explanations across different instances.

4.3.1. Effect of Neighborhood Size

Table 4 shows the effect of neighborhood size on neighborhood fidelity, stability, and computation time for the pre-hoc framework on the HELOC dataset. As the neighborhood size increases from 3 to 100, neighborhood fidelity improves from 0.8833 to 0.9670, and stability improves from 0.2152 to 0.0015, with only a minimal increase in computation time. This indicates that larger neighborhoods provide more stable and faithful explanations.

4.3.2. Computational Efficiency

Table 5 compares the computational efficiency of our frameworks with LIME on the HELOC dataset. Although our frameworks include an additional training phase, the average time to generate explanations for individual instances is significantly lower than LIME (0.011s vs. 0.3812s). This efficiency advantage becomes more apparent when generating explanations for multiple instances, with our frameworks being over 20 times faster than LIME for explaining 100 instances.

The computational efficiency of our frameworks is particularly advantageous in scenarios where real-time explanations are required or where a large number of instances need to be explained.

4.4. Qualitative Analysis of Explanations

4.4.1. Global Explanations

Figure 4 illustrates the global feature importance scores for the HELOC dataset, providing insights into the overall impact of each feature on the model’s predictions. The most influential feature is MaxDelq2PublicRecLast12M, which measures the maximum delinquency on public records in the last 12 months. This feature negatively impacts credit scores, suggesting that higher delinquency values significantly decrease the likelihood of getting a loan. Similarly, NumTrades90Ever2DerogPubRec, which represents the number of trades with derogatory public records, shows a substantial negative influence on the model’s predictions. This implies that having more trades with derogatory records decreases the probability of the target variable. The global explanation also reveals that features related to credit inquiries and satisfactory trades play a notable role in the model’s decision-making process. MSinceMostRecentInqexcl7days, indicating the time since the most recent credit inquiry, has a positive impact on the predictions, while NumSatisfactoryTrades, which represents the number of satisfactory trades, exhibits a negative influence. This suggests that recent credit inquiries and fewer satisfactory trades are associated with a higher likelihood of the target outcome.

4.4.2. Local Explanations

Figure 5 presents an example of local feature importance scores for a specific test instance from the HELOC dataset. The most influential feature for this instance is MSinceMostRecentInqexcl7days, which has a strong positive impact, indicating that a longer time since the most recent credit inquiry increases the likelihood of the target outcome for this specific instance.

Comparing the local explanation with the global explanation reveals interesting differences. Although both explanations highlight the importance of features such as MaxDelq2PublicRecLast12M and MSinceMostRecentInqexcl7days, the local explanation emphasizes features specific to the instance, such as PercentInstallTrades and NetFractionRevolvingBurden, which may not be as prominent in the global explanation. This shows the value of local explanations in capturing instance-specific factors influencing predictions.

4.5. Discussion

Our experimental results demonstrate that both the pre-hoc and co-hoc explainability frameworks outperform traditional post-hoc methods like LIME in terms of fidelity, stability, and computational efficiency. The co-hoc framework, which jointly optimizes the predictor and explainer models, consistently achieves higher fidelity scores than the pre-hoc framework, indicating better alignment between the models.

The extension to local explainability further improves the performance, with our frameworks achieving significantly higher neighborhood fidelity and stability compared to LIME. The two-phase approach, combining in-training regularization with post-hoc fine-tuning, provides both global and local explanations that are faithful to the model’s behavior. Moreover, our frameworks are computationally efficient, with significantly lower explanation generation times compared to LIME. This efficiency makes our approaches more practical for real-world applications where explanations are needed for multiple instances or in real-time.

5. Conclusions

This paper introduces two novel explainability frameworks—pre-hoc and co-hoc explainability—that integrate interpretability directly into the training process of black-box machine learning models. Unlike post-hoc methods that generate explanations after model training, our approach incorporates an inherently interpretable white-box model to guide the learning of the black-box model, ensuring that explanations are faithful to the model’s behavior without compromising accuracy. The pre-hoc framework uses a trained white-box explainer model to regularize the black-box predictor model through a fidelity term in the loss function, while the co-hoc framework jointly optimizes both models with a shared loss function. Both frameworks leverage the Jensen-Shannon divergence to measure and minimize the discrepancy between the predictions of the two models, ensuring alignment in their behaviors. We further extend these frameworks to provide local explanations by incorporating neighborhood information and developing a two-phase approach: first, training for global fidelity, then generating local explanations through fine-tuning the explainer model within the neighborhood of each test instance. This approach captures the local behavior of the black-box model, providing instance-specific explanations that are more relevant and accurate than global explanations alone.

Our experimental results on three diverse datasets—HELOC (credit risk), MovieLens (movie recommendations)- demonstrate the effectiveness of our approaches. Both frameworks achieve significantly higher fidelity than the original black-box model while maintaining comparable accuracy. The co-hoc framework consistently outperforms the pre-hoc framework in terms of fidelity, highlighting the benefits of joint optimization. When compared to the state-of-the-art post-hoc method LIME, our local explainability frameworks show superior performance in terms of point fidelity, neighborhood fidelity, and stability. Additionally, our approaches are computationally more efficient, with significantly lower explanation generation times, making them more practical for real-world applications where explanations are needed for multiple instances or in real-time. The qualitative analysis of the generated explanations reveals meaningful insights into the factors influencing the model’s predictions, both globally and locally. The global explanations provide a general understanding of feature importance across the dataset, while the local explanations capture instance-specific factors, demonstrating the value of our two-phase approach.

Several limitations and directions for future work exist. First, our current implementation focuses on binary classification tasks with tabular data. Extending the frameworks to multi-class classification, regression, and other data types (such as images and text) would broaden their applicability. Second, the choice of the white-box explainer model is limited to linear models in our experiments. Exploring other interpretable models, such as decision trees or rule-based systems, could provide alternative perspectives on model behavior.

Future research could also explore adaptive regularization schemes that dynamically adjust the trade-off between accuracy and explainability based on the complexity of the data or the model’s confidence in its predictions. In addition, incorporating user feedback into the explanation generation process could help tailor explanations to specific user needs and preferences.

In conclusion, our pre-hoc and co-hoc explainability frameworks offer a promising direction for developing machine learning models that are both accurate and transparent. By integrating explainability directly into the training process and extending it to capture local behavior, our approaches address the limitations of post-hoc methods and contribute to the advancement of trustworthy and interpretable AI systems.