Self-Explaining Reinforcement Learning for Mobile Network Resource Allocation

Konrad Nowosadko, Franco Ruggeri, Ahmad Terra · 2025

Read Self-Explaining Reinforcement Learning for Mobile Network Resource Allocation by Konrad Nowosadko, Franco Ruggeri, Ahmad Terra (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14925.txt.

Introduction

Konrad Nowosadko, Franco Ruggeri, and Ahmad Terra make reinforcement learning explain itself in a 2025 paper on mobile network resource allocation. Deep reinforcement learning performs well but hides its reasoning, which slows adoption in critical systems. Operators need to trust why a controller assigns spectrum and power the way it does.

The authors build the policy of a PPO agent from a self-explaining neural network, which produces local explanations as part of its own computation. A new aggregation method then rolls those local explanations into global ones operators can inspect. On the resource allocation task, performance lands within a small margin of the best deep learning method and well ahead of the deployed heuristic. The global explanations also agree strongly with two established explanation tools, supporting self-explaining policies for high-stakes control.

Catalog record

CREATORKonrad Nowosadko, Franco Ruggeri, Ahmad Terra
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14925

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Self-Explaining Reinforcement Learning for Mobile Network Resource Allocation Konrad Nowosadko, Franco Ruggeri, Ahmad Terra Abstract Reinforcement Learning (RL) methods that incorporate deep neural networks (DNN), though powerful, often lack transparency. Their black-box characteristic hinders interpretability and reduces trustworthiness, particularly in critical domains. To address this challenge in RL tasks, we propose a solution based on Self-Explaining Neural Networks (SENNs) along with explanation extraction methods to enhance interpretability while maintaining predictive accuracy. Our approach targets low-dimensionality problems to generate robust local and global explanations of the model’s behaviour. We evaluate the proposed method on the resource allocation problem in mobile networks, demonstrating that SENNs can constitute interpretable solutions with competitive performance. This work highlights the potential of SENNs to improve transparency and trust in AI-driven decision-making for low-dimensional tasks. Our approach strong performance on par with the existing state-of-the-art methods, while providing robust explanations. I Introduction Interest in Explainable Artificial Intelligance (XAI) has been rapidly growing, facilitated by the need for transparency. Although powerful, Deep Neural Networks (DNNs) models often operate as black boxes, making it difficult to interpret their decisions, leading to a lack of trust among stakeholders and consequently hindering their applicability. The idea behind the XAI extends onto the Reinforcement Learning (RL) through Explainable Reinforcement Learning (XRL) , which is especially important as RL systems operate in dynamic and often high-stakes environments [ 1 ] . The most commonly used explanation methods for DNNs are model-agnostic, feature-attribution methods such as LIME or SHAP, which compute feature attributions representing the impact of each feature input on the output of the model [ 2 ] . The problem of induction from XAI perspective, which states that past observations generalise on the assumption of the uniformity of the environment, discussed in [ 2 ] , shows that causal dependencies between the decision and input can be reliably inferred, given that inference data accurately reflects the deployment environment. Thus, the intrinsic XAI methods are more robust as they provide explanations derived from the model’s internal mechanism. The analogically similar division into intrinsic and post-hoc extends into XRL solutions with similar implications. Many intrinsic XRL specific methods were proposed to date [ 1 ] . Furthermore, knowing that state-of-the-art RL algorithms are model-free, the black box aspect in those methods comes from utilising black box models such as DNNs . Thus, improving the explainability of models used within the RL system can improve the explainability of the whole system. A method that combines high-capacity modelling and introduces intrinsic explainability is Self-Explaining Neural Networks (SENNs) . A type of neural network that maintains the high expressiveness of DNNs while offering an adjustable level of interpretability [ 3 ] . SENNs can be applied to the RL problem through RL algorithms, thereby constituting for the XRL solution. The main contributions of this work are as follows: • Proposing a self-explaining reinforcement learning framework that incorporates SENNs into Proximal Policy Optimisation (PPO) , a state-of-the-art RL algorithm, to achieve trustworthy predictive results with reliable explanations to solve sequential decision problems. • Modifying the architecture of SENNs to improve expressivity of the network and explainability. • Proposing a methodology to extract global explanations from the local SENNs ’ explanations. II Related work State-of-the-art RL methods are dominated by model-free algorithms such as DQN or PPO. PPO trains a stochastic policy, parametrised using DNNs . The black box character of DNNs reduces the explainability of the method. We apply SENN , a more interpretable architecture, to counteract this lack of explainability. SENN is a neural network architecture that enables intrinsic explanations, with minimal loss in predictive performance, through the notion of stability [ 3 ] . SENNs were explored in multiple works; however, to our knowledge, we are the first to apply it to the reinforcement learning problem [ 4 , 5 ] . Alternatively, model-agnostic explainability techniques such as GradSHAP, InputXGradient or DeepLift can be used to increase the explainability of DNNs based RL solutions [ 6 , 7 ] . We use them to evaluate our approach alongside the notions identified in [ 8 ] , such as cognitive relief, causality, explicitness, correctness and stability. III Background SENNs consists of three modules: conceptizer, parametrizer, and aggregator. The conceptizer acts as a feature extractor, providing the concept vector, a dimensionality-reduced input data representation [ 3 ] . The parametrizer takes in the input and produces a relevance score vector for each output class that weights each element of the concept vector. Lastly, the aggregator implements a function that aggregates the concept vector and relevance score matrix. Authors of SENNs implement an aggregator as an inner product between relevance scores and concept vectors, resulting in the final output, expressed as: \displaystyle f(x)=g(\theta(x),h(x))=\theta(x)^{T}h(x) (1) where, \theta are relevance scores matrix, h are concepts vector and g is aggregating function. Loss expression for SENN is: \mathcal{L}(x,y)=\mathcal{L}_{y}(f(x),y)+\lambda\mathcal{L}_{\theta}(f(x))+\xi\mathcal{L}_{h}(x,\hat{x}), (2) where \mathcal{L}_{\theta} is robustness loss, and \lambda is a robustness loss factor. \mathcal{L} is total loss, \mathcal{L}_{y} is classification loss, \mathcal{L}_{h} is conceptizer loss, which for the identity conceptizer is zero. Furthermore, \mathcal{L}_{\theta} in Equation ( 2 ), guarantees that relevance scores behave linearly locally, imposing local stability, an idea central to the SENNs , a measure of ensuring consistency among explanations. It can be understood as the indifference of relevance score values to small changes in the concept vector. \mathcal{L}_{\theta} is defined by: \mathcal{L}_{\theta}(f(x))=\left\|\nabla_{x}f(x)-\theta(x)^{\top}J_{x}^{h}(x)\right\|\approx 0\text{,} (3) where J^{h}_{x}(x) is a Jacobian matrix of h(x) with respect to x . For further discussion of loss terms, see [ 3 ] . Local stability is evaluated using the Lipschitz constant: L=\max_{x\in X}\frac{\|f(x)-f(x_{0})\|}{\|h(x)-h(x_{0})\|} (4) f(x) is a SENN model, h(x) is a conceptizer and L is Lipschitz constant. Lipschitz constant measures the effect of robustness loss from Equation ( 3 ). Due to the number of data samples for which evaluation is accurate and representative of the model’s behaviour, calculating the exact Lipschitz constant is time-intensive; thus, [ 3 ] authors use stochastic gradient descent to estimate it. The explainability of SENNs originates from the relevance scores. Each concept is weighted by the relevance scores, quantifying its contribution to the model’s final decision. They provide interpretability; however, it comes at the cost of performance as the local stability loss term introduced in Equation ( 3 ) is also a strong regularizer, worsening the model’s predictive performance [ 3 ] . The strength of local stability can be increased by increasing the robustness loss factor \lambda . IV Self-Explaining Reinforcement Learning In this section, first we discuss modifications we introduce to the SENNs . Furthermore, we discuss two types of explanations provided by the SENNs : local explanations, which clarify individual predictions, and global explanations, which reveal the model’s overall behaviour and how to generate both. IV-A Model architecture The SENN model we apply has two changes compared to its default form [ 3 ] . The concept explanations in [ 3 ] , which are clear for the visual data, become less explicit in non-visual domains. Due to the low-dimensional character of the approached problem, we decided to utilise an identity conceptizer, as there is no need for feature extraction. We also introduced a trainable bias vector in the aggregator to measure the model’s systematic

Read the complete text