A Comparative Study of Explainable AI Methods: Model-Agnostic vs. Model-Specific Approaches
Keerthi Devireddy · 2025
Read A Comparative Study of Explainable AI Methods: Model-Agnostic vs. Model-Specific Approaches by Keerthi Devireddy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04276.txt.
Introduction
Keerthi Devireddy compares two families of tools that explain image classifier decisions, in a 2025 paper. One family probes any model from outside. The other reads a specific network's inner activations. A standard fifty-layer network supplied the predictions. Test images ranged from dogs and birds to insects.
Outside methods gave broad feature credit that transfers across designs. Inside methods pinpointed exact activation regions more cheaply. Each family exposed different parts of the decision story. No single method won across the board. Combining methods gave the fullest picture in the tests. The stakes are highest in medicine, driving, and finance, the author notes. A selection guide matches methods to needs and budgets. Transparency, the paper argues, is a toolbox rather than a switch.
Catalog record
| CREATOR | Keerthi Devireddy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-04276
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Comparative Study of Explainable AI Methods: Model-Agnostic vs. Model-Specific Approaches Keerthi Devireddy Abstract I Introduction to Explainable AI (XAI) Introduction Deep learning models, especially convolutional neural networks (CNNs), have achieved remarkable performance across various domains, including image recognition, natural language processing, and medical diagnostics. However, their decision-making process remains a black-box, making it challenging to understand how and why certain predictions are made. Explainable AI (XAI) addresses this issue by providing interpretability to complex models, helping users, researchers, and regulatory bodies build trust in AI-driven decisions. The need for interpretability is particularly crucial in: • Healthcare: AI-based diagnostics must provide explanations to assist doctors in decision-making. • Autonomous Systems: Self-driving cars must justify their decisions for safety compliance. • Finance: Loan approvals and fraud detection systems need explainability for accountability. • Legal & Ethical AI: Ensuring AI decisions are fair, unbiased, and compliant with regulations. XAI techniques shed light on model decisions, helping identify biases, debugging models, and ensuring compliance with ethical AI guidelines. Why is XAI Important? XAI is essential for the following reasons: • Trust & Transparency – Users can trust AI-driven decisions when they are interpretable. • Debugging & Model Improvement – Helps identify errors or biases in deep learning models. • Regulatory Compliance – Ensures models adhere to AI regulations like GDPR and AI Act. • Better Human-AI Collaboration – Enables domain experts to make informed decisions based on model outputs. Mathematical Definition of Interpretability The interpretability of a model can be defined as the function: I(M)=f(M,D,C) Where: • M is the model. • D is the dataset. • C represents the constraints (such as domain knowledge, computational efficiency). • f measures the degree to which a human can understand and trust the model. A well-designed XAI method should maximize I(M) without compromising accuracy. II Model-Agnostic vs. Model-Specific XAI Overview of Explainability Methods Explainable AI (XAI) techniques can be categorized based on their dependence on the model structure. The two broad categories are: • Model-Agnostic Methods - These methods work independently of the underlying model architecture, making them flexible and applicable across different machine learning models (e.g., decision trees, neural networks, etc.). However, they often require post hoc analysis, making them computationally expensive. • Model-Specific Methods - These methods leverage the internal structure of a specific model (e.g., convolutional neural networks, decision trees) to provide more precise and architecture-dependent explanations. Feature Model-Agnostic Methods Model-Specific Methods Applicability Any ML model (CNNs, Decision Trees, etc.) Only specific models (e.g., CNNs) Flexibility High Low Computational Cost Higher due to post-hoc analysis Lower, integrated into the model Interpretability Provides broad explanations but can be less precise More precise explanations for specific architectures For this study, I have chosen two model-agnostic methods (LIME and SHAP) and two model-specific methods (Grad-CAM and Guided Backpropagation) to compare their effectiveness in explaining deep learning-based image classification. Selected XAI Methods Model-Agnostic Methods Model-agnostic methods allow us to explain any black-box model by approximating it with a simpler interpretable model or analyzing feature contributions. They are particularly useful when we do not have direct access to the internals of a model or when working with proprietary systems. 1. Local Interpretable Model-Agnostic Explanations (LIME) LIME explains individual predictions by training a simplified surrogate model that approximates the complex model’s behavior around a given instance. The key idea is to perturb the input data, observe changes in the model’s predictions, and fit a local linear model to approximate decision boundaries. Workflow of LIME: • Perturb the Input: Generate multiple variations of the input image by modifying superpixels or pixel regions. • Predict with the Model: Pass these perturbed images through the deep learning model and collect predictions. • Train a Surrogate Model: Fit a simple linear model that mimics the black-box model’s behavior around the instance of interest. • Interpretation: The coefficients of the surrogate model highlight the most important regions contributing to the prediction. Mathematical Formulation LIME optimizes the following function to ensure the surrogate model g is both faithful to the original model f and interpretable: g(x)\approx f(x) for a set of perturbed instances x^{\prime} weighted by proximity \pi_{x}(x^{\prime}) . The optimization objective for the surrogate model is: \arg\min_{g\in G}\mathcal{L}(f,g,\pi_{x})+\Omega(g) (1) where: • \mathcal{L}(f,g,\pi_{x}) is the loss function ensuring g mimics f for perturbations around x . • \pi_{x} assigns higher weights to perturbed instances closer to x . • \Omega(g) regularizes g to remain interpretable (e.g., sparsity constraint). LIME creates a local interpretable model g that approximates the complex function f in the vicinity of a given input x . The loss function \mathcal{L}(f,g,\pi_{x}) ensures that g closely resembles f , while the regularization term \Omega(g) enforces interpretability by keeping g simple (e.g., linear models or decision trees). 2. Shapley Additive Explanations (SHAP) SHAP explains model predictions using concepts from cooperative game theory, attributing feature importance based on their contribution to different coalitions of features. Unlike LIME, which approximates local behavior, SHAP ensures global and local interpretability. Workflow of SHAP: • Form Feature Coalitions: Consider different subsets of features (e.g., pixels or regions of an image). • Measure Contribution: Compute how adding or removing a feature affects model predictions. • Compute Shapley Values: Assign a score to each feature based on its average marginal contribution across all subsets. • Visualization: The output heatmap shows which regions positively or negatively influence the prediction. Mathematical Definition The Shapley value for a feature i is computed as: \phi_{i}=\sum_{S\subseteq N\setminus\{i\}}\frac{|S|!(|N|-|S|-1)!}{|N|!}\left(f(S\cup\{i\})-f(S)\right) (2) where: • S is a subset of features, • N is the set of all features, • f(S) is the model’s output when only features in S are considered. SHAP provides strong theoretical guarantees and is widely used to interpret deep learning predictions. SHAP values are derived from cooperative game theory and fairly distribute the contribution of each feature i by considering all possible subsets S of features. The term f(S\cup\{i\})-f(S) represents the change in model output when feature i is included, weighted by a factor that ensures fair allocation across different subset sizes. Model-Specific XAI Methods Model-specific methods take advantage of internal gradients and activations to provide more precise and architecture-dependent explanations. 1. Gradient-weighted Class Activation Mapping (Grad-CAM) Grad-CAM visualizes the most influential regions of an image by computing gradients of the class score w.r.t. feature maps in the CNN. It helps in localizing discriminative features used by the network. Workflow of Grad-CAM: • Forward Pass: Compute feature maps from a convolutional layer. • Compute Gradients: Calculate class-specific gradients w.r.t. these feature maps. • Weight Aggregation: Compute the importance of each feature map using global average pooling. • Heatmap Generation: Apply ReLU to generate a positive activation map overlay on the original image. Mathematical Formulation The Grad-CAM heatmap is computed as: L^{c}=ReLU\left(\sum_{k}\alpha_{k}^{c}A^{k}\right) (3) where: • A^{k} is the activation map for convolutional layer k . • \alpha_{k}^{c} represents the importance weights: \alpha_{k}^{c}=\frac{1}{Z}\sum_{i}\sum_{j}\frac{\partial y^{c}}{\partial A^{k}_{ij}} (4) • Z is the number of pixels in the feature map. Grad-CAM helps in visualizing class-discriminative regions for CNN-based models. Grad-CAM highlights image regions most relevant to a class prediction by computing the importance weights \alpha_{k}^{c} , which are obtained via gradients of the class score y^{c} with respect to feature maps A^{k} . The final heatmap is generated by applying