Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings

HALLAH SHAHID BUTT 1, QIONG HUANG 1, GÖKHAN DEMIREL 1, KEVIN FÖRDERER 1, ERFAN TAJALLI-ARDEKANI 1, SIMON WACZOWICZ 1, LUIGI SPATAFORA 1, VEIT HAGENMEYER 1, BENJAMIN SCHÄFER 1 · 2026

Read Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings by HALLAH SHAHID BUTT 1, QIONG HUANG 1, GÖKHAN DEMIREL 1, KEVIN FÖRDERER 1, ERFAN TAJALLI-ARDEKANI 1, SIMON WACZOWICZ 1, LUIGI SPATAFORA 1, VEIT HAGENMEYER 1, BENJAMIN SCHÄFER 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02049.txt.

Introduction

Butt and co-authors make deep RL for building energy use easier to trust. Homes now mix solar panels, storage, and heat pumps. Power from the sun swings through the day. Tariffs change, and more devices join the system. Control of all this grows hard to run by hand.

Deep reinforcement learning can tune building operations well. Yet its black-box nature blocks user trust and uptake. Their framework adds explanations to the RL control. It targets energy management in residential buildings. Users can see why control choices were made. That clarity aids practical adoption of the method. The work joins strong control with open reasoning. Building managers gain a tool they can audit.

Catalog record

CREATORHALLAH SHAHID BUTT 1, QIONG HUANG 1, GÖKHAN DEMIREL 1, KEVIN FÖRDERER 1, ERFAN TAJALLI-ARDEKANI 1, SIMON WACZOWICZ 1, LUIGI SPATAFORA 1, VEIT HAGENMEYER 1, BENJAMIN SCHÄFER 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.02049

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings HALLAH SHAHID BUTT 1, QIONG HUANG 1, GÖKHAN DEMIREL 1, KEVIN FÖRDERER 1, ERFAN TAJALLI-ARDEKANI 1, SIMON WACZOWICZ 1, LUIGI SPATAFORA 1, VEIT HAGENMEYER 1, BENJAMIN SCHÄFER 1 Abstract The increasing integration of renewable energy sources into power systems, particularly in buildings equipped with photovoltaic (PV) panels and energy storage systems, introduces significant complexity in energy systems. Volatile power generation, varying electricity tariffs, and increased entities, e.g., PV systems, and heat pumps, have increased the complexity and made the system harder to operate. This leads to the demand for additional control and optimization routes including data-based controls, such as reinforcement learning. While deep reinforcement learning (DRL) has emerged as a promising solution to optimize building operations in dynamic and ever more complex environments, its black-box nature impedes user trust and practical adoption. This paper presents a framework for explainable deep reinforcement learning (XRL) applied to energy management in residential buildings. We demonstrate its usage on both synthetic data but also on real-world data from the Living Lab Energy Campus (LLEC) at KIT. We train and compare both on-policy and off-policy DRL agents on an expanded state space that incorporates real-time measurements (demand, PV generation, battery power, state of charge), external signals (dynamic electricity price, local weather data), calendrical and holiday indicators, and forecasts for demand and price. Our experimental results indicate that on-policy algorithms, particularly Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), outperform off-policy methods in terms of cumulative rewards and policy stability. To explain these models, we employ post-hoc interpretation techniques to elaborate the learned control policies. Our findings demonstrate that the XRL framework not only reduces electricity costs through optimal battery management, but also provides transparent, actionable insights into the agent’s decision-making process. \NewSpotColorSpace PANTONE \AddSpotColor PANTONE PANTONE3015C PANTONE \SpotSpace 3015 \SpotSpace C 1 0.3 0 0.2 \SetPageColorSpace PANTONE † † history: Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000. † † doi: xxxx † † address: Karlsruhe Institute of Technology, 76344 Eggenstein-Leopoldshafen, Germany † † corresponding: Corresponding author: Hallah Shahid Butt (e-mail: [email protected]). I Introduction Global energy consumption has grown rapidly in recent decades due to urbanization and technological advancement [ 1 ] . Renewable and distributed energy resources (DERs) are increasingly integrated into power systems [ 2 ] . This large-scale adoption of DERs introduces new challenges: their geographical dispersion and weather dependence increase grid complexity and uncertainty [ 3 ] . For example, solar generation in Germany varies seasonally, with an average of 262 sunny hours in August compared to only 42 in December [ 4 ] . Such fluctuations highlight the need for advanced optimization and control strategies to balance supply and demand. At the building scale, the integration of local generation, on-site storage, and participation in real-time electricity markets further complicates energy management. Moreover, the growing usage of power-based devices, such as electric vehicles and heat pumps, strengthens the coupling between thermal and electrical sectors, increasing the complexity of optimization and control at both the building and grid levels. These increasing levels of complexity in building energy systems necessitate advanced control approaches capable of handling multi-domain interactions, uncertainty, and operational constraints. Model Predictive Control (MPC) has emerged as a promising strategy for such applications, offering the ability to optimize energy flows over a prediction horizon while accounting for system dynamics and forecast information [ 5 , 6 , 7 ] . In building power management, MPC enables coordinated operation of local generation, storage, and flexible loads to minimize energy costs, reduce peak demand, and improve overall efficiency [ 8 , 9 ] . Furthermore, its capability to incorporate forecasts of renewable generation, occupancy, and market signals makes MPC particularly suitable for real-time and demand-responsive energy management systems [ 10 , 11 ] . While MPC relies on explicit system models and forecasts to optimize future control actions, its performance strongly depends on model accuracy and computational tractability, particularly in large-scale and uncertain environments [ 8 , 6 ] . In contrast, Reinforcement Learning (RL) provides an AI-based control approach that learns policies directly from interaction with the environment, without requiring an explicit system model or precise forecasts [ 12 , 13 ] . By continuously adapting to changing operating conditions and uncertainties, RL can outperform MPC in scenarios where accurate models are difficult to obtain or where system dynamics evolve over time [ 14 ] . Moreover, recent studies demonstrate that RL-based controllers can effectively handle nonlinearities, stochastic disturbances, and multi-objective optimization, making them well-suited for real-time energy management in buildings with distributed energy resources and flexible loads [ 15 , 16 , 17 ] . Consequently, RL represents a promising direction for developing scalable and autonomous control frameworks that complements or even surpasses traditional MPC approaches in complex energy systems. A wide range of approaches has been proposed for energy management, from traditional controllers such as thermostats and PID regulators to embedded systems and automated devices [ 18 ] . More recently, artificial intelligence (AI) and machine learning (ML) methods, including support vector machines, random forests, and neural networks, have demonstrated significant potential in improving building energy efficiency [ 19 ] . RL has emerged as particularly promising, as it continuously interacts with the environment to derive optimal energy management policies, enabling cost-effective and adaptive control [ 20 ] . Despite this potential, ML and RL methods are often criticized for their “black-box” nature. A deeper understanding of model limitations can help AI developers enhance transparency and robustness, while increasing domain experts’ trust in model outputs and willingness to act on their recommendations. Explainable AI (XAI), particularly post-hoc explanation methods, is therefore essential for increasing transparency and facilitating adoption in energy management systems [ 21 ] . This paper addresses these explainability challenges by developing a unified framework that bridges the gap between black-box DRL control and domain-specific energy requirements. Unlike existing studies that treat explainability as an auxiliary step, this work focuses on the functional integration of post-hoc tools within the operational logic of building energy systems. Our main contributions are as follows: 1. We propose a structured and generalized integrated pipeline that links DRL policy outputs to complementary explainability mechanisms, including decision-tree–based policy approximation and feature ablation analysis, specifically adapted for high-dimensional building energy state spaces. 2. We establish a comparative evaluation protocol across multiple Deep Reinforcement Learning (DRL) algorithms (e.g., PPO, A2C), enabling consistent benchmarking under identical environmental and operational constraints. 3. We introduce a state-space design of the RL models that incorporates both historical measurements and short-term forecast information (e.g., demand and electricity price signals), with forecast horizon selection validated through quantitative error analysis to ensure operational relevance. 4. Through the integrated explainability analysis, we quantitatively identify the dominant state variables influencing policy formation and analyze how forecast information and system uncertainty shape learned control strategies. The first two points represent the methodological contributions of our work, while the third point represents the system-level contribution of the work. Lastly, one is related to insight contributions. The remainder of this paper is structured as follows: Section II reviews related works. Section III presents the methodology. Section IV discusses the results

Read the complete text