Reinforcement Learning for Robust Ageing-Aware Control of Li-ion Battery Systems with Data-Driven Formal Verification Journal: Nonlinear Analysis: Hybrid Systems

Rudi Coppola, Hovsep Touloujian, Pierfrancesco Ombrini, Manuel Mazo Jr · 2025

Read Reinforcement Learning for Robust Ageing-Aware Control of Li-ion Battery Systems with Data-Driven Formal Verification Journal: Nonlinear Analysis: Hybrid Systems by Rudi Coppola, Hovsep Touloujian, Pierfrancesco Ombrini, Manuel Mazo Jr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04288.txt.

Introduction

Rudi Coppola and three co-authors design battery charging rules that balance speed against ageing, with formal safety checks, in a 2025 research paper. Lithium-ion cells lose capacity as they age, and fast charging can hasten that loss. Battery management systems must walk that trade-off carefully. The team starts from a detailed physics-based cell model, then lets reinforcement learning agents draft controllers for different starting conditions.

A data-driven abstraction divides starting measurements into zones, each served by its own learned controller, forming a switched hybrid system. Counterexamples guide repeated redesign until criteria are met. When a design passes, the abstraction supplies probability-based guarantees for the closed loop. Safer fast-charging protocols could follow from this recipe.

Catalog record

CREATORRudi Coppola, Hovsep Touloujian, Pierfrancesco Ombrini, Manuel Mazo Jr
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-04288

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reinforcement Learning for Robust Ageing-Aware Control of Li-ion Battery Systems with Data-Driven Formal Verification Journal: Nonlinear Analysis: Hybrid Systems Rudi Coppola, Hovsep Touloujian, Pierfrancesco Ombrini, Manuel Mazo Jr Abstract Rechargeable lithium-ion (Li-ion) batteries are a ubiquitous element of modern technology. In the last decades, the production and design of such batteries and their adjacent embedded charging and safety protocols, denoted by Battery Management Systems (BMS), has taken central stage. A fundamental challenge to be addressed is the trade-off between the speed of charging and the ageing behavior, resulting in the loss of capacity in the battery cell. We rely on a high-fidelity physics-based battery model and propose an approach to data-driven charging and safety protocol design. Following a Counterexample-Guided Inductive Synthesis scheme, we combine Reinforcement Learning (RL) with recent developments in data-driven formal methods to obtain a hybrid control strategy: RL is used to synthesise the individual controllers, and a data-driven abstraction guides their partitioning into a switched structure, depending on the initial output measurements of the battery. The resulting discrete selection among RL-based controllers, coupled with the continuous battery dynamics, realises a hybrid system. When a design meets the desired criteria, the abstraction provides probabilistic guarantees on the closed-loop performance of the cell. 1 Introduction The development of advanced battery control strategies is a critical enabler for the widespread electrification of both energy systems and transportation. These strategies must ensure the safe and reliable operation of batteries under a wide range of conditions throughout their service life. At the same time, growing economic pressures and sustainability goals demand more efficient use of battery systems—minimizing degradation while maximizing performance [ 1 ] . Whether in grid-scale storage or electric vehicles, improving how batteries are charged and managed is central to achieving these objectives. To this end, significant progress in Battery Management Systems (BMS) is required. Conventional BMS implementations rely heavily on Equivalent Circuit Models (ECMs), consisting of circuital models describing the battery as a set of simplified electrical components. Although ECMs offer fast computation and are widely used in practice, they fail to capture many internal electrochemical processes of the battery. As a result, they are limited in their ability to estimate internal states or inform control strategies that account for degradation mechanisms. In contrast, physics-based electrochemical models, typically framed as Doyle-Fuller-Newman (DFN) models, offer a more detailed description of the internal dynamics governing battery behavior. These models can, in principle, enable estimation and control of internal variables that are otherwise unobservable, potentially leading to smarter, degradation-aware charging protocols. However, the high computational complexity of DFN models hinders its application in online optimization-based approaches, e.g. nonlinear Model Predictive Control (MPC), or to develop a formal feedback control design. To bridge this gap, we explore a Reinforcement Learning (RL) approach that leverages the accuracy of physics-based models to construct a light-weight controller by interacting offline with a DFN model simulator. We propose an automatic model-free RL framework that learns optimal charging strategies through interaction with a simulated environment governed by the full electrochemical model. While training relies on DFN simulations, the final controller relies solely on measurable quantities, such as voltage and temperature, rendering our scheme easily applicable in practice. In parallel, we apply a modern statistical tool to provide safety and performance guarantees of the closed-loop system. This approach offers a promising path toward advanced battery control that is both physically grounded and practically implementable. A lithium-ion (Li-ion) battery consists of a negative electrode (anode), a separator, and a positive electrode (cathode). The electrodes are porous, composed by microscopic particles, and immersed in a ion-conducting liquid electrolyte. Charging occurs as the applied external current forces the internal movement of the ions from the cathode to the anode. During discharging the process is reversed. While this mechanism is highly reversible, enabling hundreds to thousands of charge–discharge cycles, the long-term performance of a battery is constrained by irreversible ageing mechanisms that accumulate over time [ 2 ] . One of the most prominent degradation processes is the growth of the solid–electrolyte interphase (SEI) layer on the surface of the negative electrode. The SEI layer forms as a result of electrolyte decomposition during the initial cycles and continues to grow during subsequent operation due to parasitic side reactions. As the SEI thickens, it increases interfacial resistance and consumes active lithium, effectively reducing the battery’s usable capacity and power capability [ 3 ] . The kinetics of SEI growth have been modeled as an electron-limited process driven by side reactions at the anode–electrolyte interface, with foundational work provided by Darling and Newman [ 4 ] . Importantly, SEI growth is strongly correlated with charging rates: faster charging typically leads to increased SEI formation due to elevated reaction rates and higher overpotentials. This creates an inherent trade-off in charging strategies between minimizing charge time and limiting long-term degradation. Related work: Recent studies have begun to address this by developing ageing-aware charging protocols that balance performance and longevity using physics-based models in a control context, as done in [ 2 ] . Ageing-aware MPC schemes are implemented on the DFN model, such as in [ 5 , 6 ] , where an approximate model is used to describe the internal battery dynamics. This work is expanded in [ 7 ] , where a nonlinear MPC framework is implemented using nonlinear optimisation methods. However, the control scheme developed is implemented in full-state feedback as no state estimator concept is developed. As a result, the corresponding experimental runs are performed in an open-loop environment which is detrimental to the charging performance and safety. Additionally, the computational difficulty of MPC based on the DFN model is illustrated in [ 8 ] , where several problem formulation strategies are considered for the DFN model, each resulting in substantially large time required for a single optimization call (45-65 seconds), which renders real-time MPC on such a complex model infeasible in most practical applications. Reinforcement Learning (RL) is a model-free approach to policy design. Such a policy, or controller, is designed in [ 9 ] , where the DFN Model is used to emulate a cylindrical cell Li-ion Battery System, and a controller is trained for the case of both state feedback and output feedback, i.e., the controller is assumed to only know the cell voltage, the average temperature of the cell, and its State of Charge (SOC). The resulting protocol is shown to perform better than standard rule-based charging methods. The authors model ageing as lithium plating [ 10 , 11 ] ; accordingly, the cost function does not aim at extending the life cycle of the battery, rather it aims at fast charging while avoiding destructive phenomena. In our work, we model ageing as the SEI growth causing the decay of the State of Health (SOH) of the cell, a measure of its capacity; we explicitly incorporate in the cost function of the RL problem a metric for capacity loss. In [ 12 ] the authors apply RL to the problem of safe fast charging and augment it with a safety layer that projects unsafe actions into a feasible region defined by a Gaussian Process (GP) surrogate model. In contrast with our work, the authors do not model ageing constraints.

Read the complete text