LLEXICORP: End-user Explainability of Convolutional Neural Networks

Vojtěch Kůr, Adam Bajger, Adam Kukučka, Marek Hradil, Vít Musil, Tomáš Brázdil · 2025

Read LLEXICORP: End-user Explainability of Convolutional Neural Networks by Vojtěch Kůr, Adam Bajger, Adam Kukučka, Marek Hradil, Vít Musil, Tomáš Brázdil (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-02720.txt.

Introduction

Explaining why a vision network labels a photo is hard for non-experts. Vojtech Kur and five colleagues built LLEXICORP, a 2025 pipeline that explains convolutional networks in plain words. It builds on concept relevance propagation, a method that traces a prediction back to channels in the top layers and finds images that excite each channel most.

Older workflows made experts name those concepts and write the explanations by hand. LLEXICORP instead hands the task to a multimodal language model, with prompts that keep naming and explaining apart for faithfulness. The text can be pitched low for experts or high for lay readers. Tests on ImageNet photos with a VGG16 model suggest the pairing can open up network decisions to far more people.

Catalog record

CREATORVojtěch Kůr, Adam Bajger, Adam Kukučka, Marek Hradil, Vít Musil, Tomáš Brázdil
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-02720

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLEXICORP: End-user Explainability of Convolutional Neural Networks Vojtěch Kůr, Adam Bajger, Adam Kukučka, Marek Hradil, Vít Musil, Tomáš Brázdil Abstract Convolutional neural networks (CNNs) underpin many modern computer vision systems. With applications ranging from common to critical areas, a need to explain and understand the model and its decisions (XAI) emerged. Prior works suggest that in the top layers of CNNs, the individual channels can be attributed to classifying human-understandable concepts. Concept relevance propagation (CRP) methods can back‑track predictions to these channels and find images that most activate these channels. However, current CRP workflows are largely manual: experts must inspect activation images to name the discovered concepts and must synthesize verbose explanations from relevance maps, limiting the accessibility of the explanations and their scalability. To address these issues, we introduce Large Language model EXplaIns COncept Relevance Propagation (LLEXICORP), a modular pipeline that couples CRP with a multimodal large language model. Our approach automatically assigns descriptive names to concept prototypes and generates natural‑language explanations that translate quantitative relevance distributions into intuitive narratives. To ensure faithfulness, we craft prompts that teach the language model the semantics of CRP through examples and enforce a separation between naming and explanation tasks. The resulting text can be tailored to different audiences, offering low‑level technical descriptions for experts and high‑level summaries for non‑technical stakeholders. We qualitatively evaluate our method on various images from ImageNet on a VGG16 model. Our findings suggest that integrating concept-based attribution methods with large language models can significantly lower the barrier to interpreting deep neural networks, paving the way for more transparent AI systems. Introduction Convolutional neural networks (CNNs) remain a cornerstone of modern computer vision ( LeCun, Bengio, and Hinton 2015 ; Rawat and Wang 2017 ) , powering applications that range from everyday tasks, such as image classification on mobile devices, to safety‑critical domains like medical imaging ( Litjens et al. 2017 ) and autonomous driving ( Grigorescu et al. 2020 ) . As these models increasingly influence high‑stakes decisions, the demand for interpretable and trustworthy explanations of their predictions has grown ( Huang et al. 2018 ) . This need has fueled the development of Explainable AI (XAI), which seeks to make the inner workings and decision rationales of complex models more transparent to both experts and non‑experts ( Samek, Wiegand, and Müller 2017 ) . In computer vision, explainability has traditionally been approached by identifying regions of an input image that are most significant for a model’s decision. Various techniques have been proposed to determine this significance (e.g., Grad-CAM ( Selvaraju et al. 2017 ) , SHAP ( Lundberg and Lee 2017 ) , LIME ( Ribeiro, Singh, and Guestrin 2016 ) , or D-RISE ( Petsiuk et al. 2021 ) ). Due to limitations of saliency-based explanations ( Kindermans et al. 2019 ) , recent research in CNN interpretability emphasizes understanding the higher‑level representations learned by the network. Studies have shown that neurons or channels in the deeper layers (i.e., layers close to the output) of CNNs often correspond to semantically meaningful and human‑understandable concepts ( Bau et al. 2017 ; Zhou et al. 2018 ) . For instance, a specific feature channel might respond strongly to a cat’s tail, while another might specialize in detecting the contour of its legs. This concept‑level perspective supports explanations that align more naturally with human reasoning. A wide range of research in model interpretability focuses on bridging the gap between human‑understandable concepts and the internal representations of neural networks. Popular methods include Concept Activation Vectors (TCAV) ( Kim et al. 2018 ) and ACE ( Ghorbani et al. 2019 ) . A wide range of research in model interpretability focuses on bridging the gap between human‑understandable concepts and the internal representations of neural networks. Two complementary directions have emerged. The first is concept attribution , which seeks to identify which neurons, channels, or feature maps correspond to semantic concepts. Popular methods include Concept Activation Vectors (TCAV) ( Kim et al. 2018 ) and ACE ( Ghorbani et al. 2019 ) . The second direction is relevance propagation , which identifies which internal components and input regions contribute most to a single prediction. Methods such as Layer‑wise Relevance Propagation (LRP) ( Bach et al. 2015 ; Montavon et al. 2019 ) , DeepLIFT ( Shrikumar, Greenside, and Kundaje 2017 ) , and Integrated Gradients ( Sundararajan, Taly, and Yan 2017 ) propagate the prediction score backward through the network to produce fine‑grained attributions. In this work, we built upon Concept Relevance Propagation (CRP) ( Achtibat et al. 2023b ) . In CRP language, a concept is both the semantic concept a channel recognizes and the channel itself. For a single prediction, CRP finds the most important components for the prediction using LRP. Then, the concept is visualized by finding images for which the channel is most relevant during their prediction. While CRP has demonstrated strong potential for interpreting model behavior, it suffers from two key limitations. First, the workflow relies on human experts to manually inspect activation images and assign meaningful names to discovered concepts. Second, translating the resulting relevance maps into coherent explanations requires significant domain expertise, making the method inaccessible to non‑experts and difficult to scale for large‑scale deployment. Addressing these limitations requires a scalable approach to convert raw concept relevance into coherent, faithful narratives—a challenge well suited to the emerging capabilities of large language models (LLMs). To address these challenges, we propose Large Language model EXplaIns COncept Relevance Propagation (LLEXICORP), a modular framework that integrates CRP with a multi‑modal LLM. LLEXICORP automatically translates complex relevance maps and concept prototypes into clear, human‑understandable textual explanations while faithfully preserving the underlying CRP reasoning. The framework supports multiple levels of abstraction, providing technical insights for experts and accessible summaries for non‑specialists, thereby broadening the reach of concept‑based XAI. Our approach relies on carefully designed prompt engineering. By supplying the LLM with illustrative examples of CRP outputs, we guide it to produce accurate and coherent narratives. Moreover, by isolating the concept‑naming and explanation‑generation stages, we maintain faithfulness to the original attribution method while ensuring that each explanation remains interpretable and scalable. Contribution In summary, this work (i) identifies the key bottlenecks of CRP for scalable interpretability, (ii) introduces LLEXICORP, a hybrid CRP+LLM framework for automated concept-level explanations, and (iii) demonstrates how prompt-engineered LLMs can make CNN reasoning more accessible to both experts and non-experts. Figure 1: Exaple image of lizard from ImageNet. Figure 2: The activation heatmaps of the five most relevant convolutional filters, as found by CRP. Figure 3: Top 6 images that most activate on each of the 5 most relevant concepts, as found by CRP, together with a saliency map of activation of the said concepts. The model predicts the image as an “American chameleon” with 86.11% confidence, supported by five key concepts. The most influential concept (40.62%) is “lizards,” directly recognized in the lizard’s body, tail, and limbs, which are essential features for identifying the species. The second concept (21.35%) is “elongated shapes with distinct edges,’ recognized in the lizard’s elongated body, head, torso, and tail, contributing structurally to the prediction. The third concept (21.29%) is “elongated green leaves or stems,” directly recognized in the surrounding foliage, which

Read the complete text