Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems
Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank, Ralf Mikut, Veit Hagenmeyer · 2026
Read Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems by Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank, Ralf Mikut, Veit Hagenmeyer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26249.txt.
Introduction
Henrich and co-authors shrink transformer control policies for home energy systems so they can run on weak hardware. A Decision Transformer learns battery dispatch from past data, raising solar self-use and cutting costs. The team distils large teacher models into small students that copy their actions. Students keep control quality, and some improve cost results by up to 1%. At the same time they cut parameters by up to 96% and run up to 63% faster.
Catalog record
| CREATOR | Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank, Ralf Mikut, Veit Hagenmeyer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53569 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26249
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank, Ralf Mikut, Veit Hagenmeyer Abstract . Transformer-based reinforcement learning has emerged as a strong candidate for sequential control in residential energy management. In particular, the Decision Transformer can learn effective battery dispatch policies from historical data, thereby increasing photovoltaic self-consumption and reducing electricity costs. However, transformer models are typically too computationally demanding for deployment on resource-constrained residential controllers, where memory and latency constraints are critical. The present paper investigates Knowledge Distillation to transfer the decision-making behaviour of high-capacity Decision Transformer policies to compact models more suitable for embedded deployment. Using the Ausgrid dataset, we train teacher models in an offline, sequence-based Decision Transformer framework on heterogeneous multi-building data. Given the trained models, we distil smaller student models by matching the teachers’ actions, thereby preserving control quality while reducing model size. Across a broad set of teacher–student configurations, distillation largely preserves control performance and even yields small improvements of up to 1 % 1\text{\,}\mathrm{\%} , while reducing the parameter count by up to 96 % 96\text{\,}\mathrm{\%} , the inference memory by up to 90 % 90\text{\,}\mathrm{\%} , and the inference time by up to 63 % 63\text{\,}\mathrm{\%} . Beyond these compression effects, comparable cost improvements are also observed when distilling into a student of identical architectural capacity. Overall, our results show that Knowledge Distillation makes Decision Transformer control more applicable for residential energy management on resource-limited hardware. Figure 1 . Graphical Abstract 1. Introduction The increasing deployment of renewable energy sources, particularly Photovoltaic (PV) systems, introduces significant variability and temporal imbalances between electricity generation and household demand. To address these fluctuations, Home Energy Management Systems (HEMSs) have become critical for coordinating flexible resources, such as Battery Energy Storage Systems (BESSs) , to lower operational costs and improve grid stability ( 21 ) . While traditional methods such as rule-based control ( 14 ; 1 ) or Model Predictive Control (MPC) ( 38 ; 30 ) are often used, they frequently struggle with highly variable conditions or rely on system models that are difficult to maintain in real-world settings ( 39 ) . As a data-driven alternative, Reinforcement Learning (RL) has gained prominence, with the Decision Transformer (DT) ( 6 ) emerging as a novel architecture that frames decision-making as a conditional sequence-modelling problem. By utilising a Transformer-based decoder to predict actions from past observations and target returns, the DT provides a stable, goal-conditioned framework. However, the performance potential of Transformer architectures is fundamentally tied to their considerable model sizes, which can range from hundreds of millions to several billions of parameters, and to the amount of training data ( 17 ) . Empirical evidence further indicates that increasing model capacity typically leads to improved performance ( 15 ) . At the same time, the resulting computational and memory demands pose a significant challenge for deployment on resource-constrained embedded hardware used in residential energy controllers. To bridge the gap between architectural complexity and practical deployment, Knowledge Distillation (KD) , introduced as a formal term by 13 , serves as a primary strategy for model compression. KD employs a teacher-student paradigm, where a high-capacity teacher model transfers its learned behavioural patterns to a smaller, more efficient student. This process allows the student to mimic the teacher’s internal representations, enabling the creation of compact models that preserve predictive control without the massive hardware requirements of the original architecture ( 11 ) . Building on this principle, this work explores the application of KD to DTs trained on aggregated multi-building datasets to leverage the needed diverse structural patterns and enhance policy robustness. 1.1. Related Work Advances in computer vision ( 8 ; 36 ) and natural language processing ( 26 ; 35 ) have established KD as a key technique for transferring the capabilities of large, high-performing models into compact architectures. Building on these successes, recent research has increasingly explored the application of KD in energy systems, where similar efficiency–performance trade-offs arise in edge-deployed controllers and forecasting pipelines. Accordingly, this section reviews recent applications of KD across load identification, energy forecasting, battery health management, and distributed and RL -based energy systems. Furthermore, we explore the domain of DTs , review two papers combining them with KD , and conclude with a discussion of the role of DTs in energy systems. Knowledge Distillation in Energy In the domain of non-intrusive load monitoring, KD is commonly employed to transfer knowledge from high-accuracy, cloud-based teacher models to lightweight student models deployed on smart meters: 37 propose a one-dimensional CNN–BiLSTM–attention teacher model to guide the training of a compact student architecture, achieving a balance between identification accuracy and local computational constraints. To further reduce model complexity, 41 introduce an integrated pruning and distillation framework, IPD-NILM, which combines layer-wise iterative Fisher pruning with distillation and reduces the number of trainable parameters by 72\text{\,}\mathrm{\%} while maintaining comparable performance. Moving beyond compression alone, 3 incorporate perception-aligned gradients into the distillation process to ensure that student models inherit reliable and human-interpretable decision patterns, addressing the opacity of deep neural networks deployed at the edge. Beyond load identification, KD has also been applied to energy forecasting and battery health management in order to enable the local deployment of complex time-series and graph-based models. 4 demonstrate the distillation of large Transformer-based foundation models into compact expert models for battery capacity degradation forecasting, enabling cross-capacity generalisation suitable for onboard battery management systems. Similarly, 20 distil spatial–temporal knowledge from graph neural networks pretrained on utility-scale datasets into household-level multilayer perceptrons, alleviating data scarcity at the local level. In the context of dynamic multi-energy microgrids, 2 employ KD-LSTM architectures to adapt to evolving operating conditions, extracting salient features from sparse data while significantly reducing computational resource consumption. In distributed learning environments, particularly federated learning, KD has been shown to mitigate challenges arising from non-independent and identically distributed data and heterogeneous hardware capabilities. In the context of short-term forecasting in electric Internet-of-Things systems, 31 propose hierarchical KD to identify and distil knowledge from a subset of top-performing clients that best represent the global gradient direction, improving convergence stability in electric Internet-of-Things systems. For energy theft detection in smart grids, 42 introduce an edge-assisted federated contrastive distillation approach in which local teacher models guide student training, reducing detection loss without requiring centralised data aggregation. KD has also been explored in RL –based energy management to enhance policy interpretability. In the domain of building energy management, 5 propose value-aware distillation to extract interpretable decision rules from complex Soft Actor-Critic policies into decision tree ensembles, preserving over 91\text{\,}\mathrm{\%} of the teacher’s performance while improving transparency for system operators. Knowledge Distillation and Decision Transformer In the domain of offline multi-agent RL , 32 train a centralised DT teacher with access to all agents’ observations, actions, and rewards, and subsequently distil both action predictions and inter-agent relational structure into decentralised student policies, yielding improved convergence and robustness over sequence-modelling and offline RL baselines. In the context of visual robotic manipulation, 7 propose a transformer-based KD framework in which a state-based decision-transformer teacher transfers control policies to an image-based student via state estimation and weight initialisation, enabling effective policy learning under partial observability. Decision Transformer