Readable Twins of Unreadable Models

Krzysztof Pancerz, Piotr Kulicki, Michał Kalisz, Andrzej Burda, Maciej Stanisławski, Jaromir Sarzyński · 2025

Read Readable Twins of Unreadable Models by Krzysztof Pancerz, Piotr Kulicki, Michał Kalisz, Andrzej Burda, Maciej Stanisławski, Jaromir Sarzyński (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13150.txt.

Introduction

Krzysztof Pancerz and five co-authors write in 2025 about making deep learning models easier to explain. Trust in AI systems depends in part on whether people can follow how a model reaches its call. Deep models are often too complex to read directly.

The authors borrow the twin idea from work on physical objects. They build a readable twin of a deep model, in the form of a rough flow model that people can follow. The paper lays out the full steps for the switch from one form to the other. A worked case sorts handwritten digits from a well-known data set to show the approach in action.

Catalog record

CREATORKrzysztof Pancerz, Piotr Kulicki, Michał Kalisz, Andrzej Burda, Maciej Stanisławski, Jaromir Sarzyński
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE10 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-13150

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Readable Twins of Unreadable Models Krzysztof Pancerz, Piotr Kulicki, Michał Kalisz, Andrzej Burda, Maciej Stanisławski, Jaromir Sarzyński Abstract Creating responsible artificial intelligence (AI) systems is an important issue in contemporary research and development of works on AI. One of the characteristics of responsible AI systems is their explainability. In the paper, we are interested in explainable deep learning (XDL) systems. On the basis of the creation of digital twins of physical objects, we introduce the idea of creating readable twins (in the form of imprecise information flow models) for unreadable deep learning models. The complete procedure for switching from the deep learning model (DLM) to the imprecise information flow model (IIFM) is presented. The proposed approach is illustrated with an example of a deep learning classification model for image recognition of handwritten digits from the MNIST data set. 1 Introduction One of the most current problems in Artificial Intelligence (AI) is to make AI tools human-readable (interpretable, explainable, etc.) and in consequence to make AI responsible (cf. [ 1 ] ). AI tools are currently, in many cases, reinforced by deep neural networks (DNNs). Therefore, special attention in research on Explainable Artificial Intelligence (XAI) is focused on Explainable Deep Learning (XDL) (cf. [ 11 ] ). The taxonomy of the trends identified for explainability techniques related to Deep Learning Models (DLMs) distinguishes, among others, model-agnostic techniques (MATs) and model-specific techniques (MSTs) (cf. [ 1 ] ). An explainer in MATs is capable of explaining any model (cf. the LIME technique [ 12 ] ). An explainer in MSTs is correlated with a given deep learning model (cf. the DeepLIFT technique [ 13 ] ). In our research, we are developing a new technique that can be classified as MST. This technique is provided with an acronym HuReTEx (Human Readable Twin Explainer for Deep Learning Models). An idea of transformation of a deep learning model (DLM) into imprecise information flow model (IIFM) via a Sequential Information System (SIS) is shown in Figure 1 . In this transformation, DLM is original, numerical and machine-readable, while IIFM is a twin of DLM that is symbolical and human-readable. Figure 1: An idea of creation of a readable twin (an imprecise information flow model) for an unreadable original model (a deep learning model). Figure 2: A flowchart of the transformation procedure of DLM into IIFM. HuReTEx can be treated as a reference to the ideas of mirror worlds (cf. [ 4 ] ), as well as digital twins (cf. [ 7 ] ), as it is shown in Table 1 . Moreover, the proposed approach refers to twin-systems for XAI (cf. [ 5 ] ). For a model that is unreadable to humans, its readable twin is built. Model Description Digital twin model A combination of a physical object and its digital representation in virtual space. Readable twin model A combination of an unreadable deep learning model and its readable representation. Table 1: Digital twin model vs. readable twin model. Transformation is preformed in several main stages (see a flowchart in Figure 2 ). Rough set flow graphs (RSFGs) [ 10 ] (used as IIFMs) and triangular norms or co-norms together with evolutionary algorithms can be used to mine and visualize the most confident prediction paths in RSFGs explaining decisions proposed by DLMs (see next section). In our approach, the twin model is built on the basis of aggregated artifacts (at specific levels of abstraction) generated by individual model layers for training data. In this case, the explanations given by the model are not generated for individual input cases only. 2 From Unreadable Models to Their Readable Twins In this section, the consecutive steps of the procedure to create a readable twin of an unreadable deep learning model are described. A readable twin model has the form of a rough set flow graph (RSFG). Elements of RSFG correspond to the original deep learning model (DLM) as it is shown in Table 2 . Deep Learning Model (DLM) Rough Set Flow Graph (RSFG) A convolutional layer of DLM A node layer of RSFG Clusters of artifacts generated by filters Nodes in a layer of RSFG of a convolutional layer of DLM A dense layer of DLM A node layer of RSFG Clusters of artifacts generated by neurons Nodes in a layer of RSFG of a dense layer of DLM Table 2: DLM vs. RSFG. It is worth noting that we have omitted the input layer and the flatten layer because they are not trained in deep learning models and therefore they do not acquire knowledge that could be used in the explanation process. An illustrative example is presented for a simple deep learning model built using the Keras library [ 2 ] to classify images from the well-known MNIST database of handwritten digits (see Figure 3 ) [ 3 ] . The training data set includes 48000 images. Figure 3: A fragment of the MNIST data set (by Suvanjanprasai - Own work, CC BY-SA 4.0 https://commons.wikimedia.org/w/index.php?curid=156115980). Step 1: Building, training and testing a deep learning model (DLM) on training cases (images). The model can be built (see Figure 4 ), trained and tested in a standard way. Figure 4: Creation of a readable twin for an unreadable original model: Step 1. Step 2: Calculating and clustering activations generated by key layers of DLM for training cases (images). After training and testing DLM on images from the MNIST data set, the activations of the model entities for each image from the training data set are calculated and clustered. A good choice may be agglomerative clustering leading to a hierarchical structure of clusters, which can be easily mapped to a hierarchical conceptual knowledge structure about artifacts. In an example in Figure 5 , the cluster identifiers, to which artifacts generated for images from the training data set belong, are shown. Figure 5: Creation of a readable twin for an unreadable original model: Step 2. Step 3: Creating a sequential information system (SIS). The underlying information for generation of a rough set flow graph is arranged in the form of a sequential information system (see an example in Figure 6 ), that is, an information system (called in [ 9 ] a knowledge representation system) with an ordered set of attributes (presented in columns). For convolutional layers, attribute values are tuples of the identifiers of clusters obtained in Step 2. For dense layers, attribute values are identifiers of clusters obtained in Step 2. For an output layer, attribute values are labels assigned to training cases (images). Figure 6: Creation of a readable twin for an unreadable original model: Step 3. Step 4: Creating a rough set flow graph (RSFG). RSFG has a layered structure. Each layer corresponds to one attribute (column) of SIS obtained in Step 3. The nodes in a given layer represent the values of a given attribute of SIS. An imprecise information flow between nodes is described by three coefficients (certainty, strength, and covering) assigned to edges connecting nodes. The certainty of a given edge connecting nodes n and n^{\prime} determines how many times a transition occurs from node n to node n^{\prime} relative to all transitions from node n to other nodes. The covering of

Read the complete text