Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks

Sebastian Seidel, Uwe M. Borghoff · 2025

Read Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks by Sebastian Seidel, Uwe M. Borghoff (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12446.txt.

Introduction

Sebastian Seidel and Uwe Borghoff worked in 2025 on turning neural networks into symbol models that people can read. Neural networks are powerful but opaque, which limits trust. Decision trees, by contrast, show their logic openly while keeping the same choices.

Their method traces neuron values and inputs layer by layer in a feedforward network. Those traces map onto the edges of a decision tree, using parts such as fillers and roles in the network's spread-out codes. Deeper networks are handled by refining subpaths one hidden layer at a time. A prototype built on standard tools showed the extraction works in practice, aiding trust and accountability.

Catalog record

CREATORSebastian Seidel, Uwe M. Borghoff
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12446

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks Sebastian Seidel, Uwe M. Borghoff Abstract Artificial intelligence (AI) has emerged as a transformative force across industries, driven by advances in deep learning and natural language processing, and fueled by large-scale data and computing resources. Despite its rapid adoption, the opacity of AI systems poses significant challenges to trust and acceptance. This work explores the intersection of connectionist and symbolic approaches to artificial intelligence, focusing on the derivation of interpretable symbolic models, such as decision trees, from feedforward neural networks (FNNs). Decision trees provide a transparent framework for elucidating the operations of neural networks while preserving their functionality. The derivation is presented in a step-by-step approach and illustrated with several examples. A systematic methodology is proposed to bridge neural and symbolic paradigms by exploiting distributed representations in FNNs to identify symbolic components, including fillers, roles, and their interrelationships. The process traces neuron activation values and input configurations across network layers, mapping activations and their underlying inputs to decision tree edges. The resulting symbolic structures effectively capture FNN decision processes and enable scalability to deeper networks through iterative refinement of subpaths for each hidden layer. To validate the theoretical framework, a prototype was developed using Keras .h5 -data and emulating TensorFlow within the Java JDK / JavaFX environment. This prototype demonstrates the feasibility of extracting symbolic representations from neural networks, enhancing trust in AI systems, and promoting accountability. † † impactstatement: This work addresses the pressing need for confidence and transparency in AI systems by bridging connectionist and symbolic approaches. By providing a clear methodology to demystify the “black box” nature of AI, it empowers stakeholders—users, developers, and regulators—to better understand and trust AI decisions and the “rational” actions they produce. The research advances Explainable AI (XAI) by contributing both theoretical insights and practical tools for interpretability, facilitating ethical and effective AI deployment. This innovation is particularly impactful in sensitive fields such as healthcare, finance, and autonomous systems, where transparency is critical to ensuring accountability and public trust. By fostering greater acceptance of AI technologies, this work accelerates responsible AI integration, shaping a future where innovation aligns with societal needs for ethical decision-making and reliable systems. I Introduction AI has become a focal point of attention, often described as experiencing a “hype.” This surge of interest is driven by its transformative potential across industries, promising significant efficiency gains and driving innovation, particularly in the areas of image recognition and analysis and natural language processing. The proliferation of large datasets, coupled with increasingly powerful computational resources, has further accelerated the development and deployment of AI applications. As a result, AI has become a cornerstone of modern IT systems, embedded in everything from consumer products to large-scale industrial processes. Despite these advancements, a significant gap remains in terms of trust [ 1 ] and acceptance for AI systems, particularly those perceived as “black boxes” [ 2 ] . This lack of trust has led to a growing interest in XAI. The core goal of XAI is to make AI systems more transparent and interpretable by providing understandable explanations for their decisions and actions [ 3 ] . By addressing the opaque nature of many AI algorithms, XAI aims to bridge the gap between the technical intricacies of machine learning and the human need for understanding. Transparency in AI is not only a technical challenge, but also a social and ethical imperative [ 4 ] . Stakeholders need to trust AI systems to interact effectively with them. This is especially critical in sensitive areas where the stakes of incorrect or misunderstood AI decisions are high, such as military applications [ 5 , 6 ] . It is also important to prevent AI from being increasingly integrated into criminal and harmful activities and going undetected [ 7 ] . The present work contributes to this field by offering a systematic categorization and analysis of symbolic and connectionist approaches within AI. Symbolic approaches are grounded in explicit, rule-based reasoning, while connectionist approaches, exemplified by artificial neural networks, rely on distributed representations and statistical learning. This paper not only delineates the relationships, distinctions, and respective advantages of these approaches but also demonstrates the value of deriving equivalent symbolic decision models from neural models. Such derivations enhance transparency and interpretability without compromising the functionality of the underlying AI systems [ 8 ] . A special focus is placed on feedforward neural networks [ 9 ] , including similar types of neural networks that can be represented in a feedforward form, such as convolutional neural networks. Recurrent architectures are not explicitly considered in this study. The analysis underscores that symbolic models such as decision trees, finite state machines (for classification or as transducers), and behavior trees are viable candidates for representing the decision logic of neural networks. Among these, decision trees are identified as the most appropriate symbolic counterpart due to their fit with internal processes, interpretability, and alignment with the explainability goals of XAI [ 10 , 11 ] . This paper thus situates itself within the broader discourse on enhancing AI explainability by bridging the gap between connectionist models and symbolic reasoning. By doing so, it contributes not only to the technical development of interpretable AI systems but also to their societal acceptance and ethical deployment. In the following sections, we delve deeper into the methodologies and findings that underpin this contribution, offering insights into the practical and theoretical implications of deriving symbolic representations for neural networks. This work serves as a step forward in the journey toward making AI not just powerful but also accountable and understandable. The paper is organized as follows. Sect. II introduces basic concepts and reviews key related work in the field, providing both a solid theoretical framework and practical context. Sect. III outlines the core contribution of this work, detailing the methodology for deriving decision trees from feedforward neural networks (FNNs). The process is demonstrated with clear examples and supported by suitable data structures, broken down into simple, easy-to-follow steps. Notably, the final derivation procedure is also applicable to decision paths in deep feedforward neural networks, ensuring scalability and versatility. Sect. IV presents a proof-of-concept prototype capable of managing larger FFNs and CNNs with their components, including different layers, pooling mechanisms, and the hierarchical structure of the resulting decision tree. Additionally, a brief overview of the graphical user interface is provided. Sect. V summarizes the procedure for deriving a hierarchical decision tree from a feedforward neural network, highlighting the creation of decision paths for input vectors and their combination into a unified tree, showcasing how symbol-based models make the neural network’s inner workings transparent. II Basics and Related Work Artificial intelligence as a generic term for automated, self-optimizing and similar systems should serve as a starting point for the necessary classification of our subject of investigation. According to [ 12 ] , the field of artificial intelligence can be divided into the following four categories based on their respective goals: • realization of human thought • generating human actions • realization of rational thought • generating rational actions To achieve the last goals, the field of artificial intelligence offers two basic approaches, which are partly contradictory in the way they work. Their

Read the complete text