C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents

Guihlerme Daubt 1 and Adrian Redder 2 1 Faculty of Technology, Bielefeld University, Germany 2 Department of Electrical Engineering and Information Technology, Paderborn University, Germany [email protected] , [email protected] · 2026

Read C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents by Guihlerme Daubt 1 and Adrian Redder 2 1 Faculty of Technology, Bielefeld University, Germany 2 Department of Electrical Engineering and Information Technology, Paderborn University, Germany [email protected] , [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24241.txt.

Introduction

Daubt and Redder present C-STEP, a safety measure for physics-informed reinforcement learning in mobile robots. Safe navigation in complex settings is still hard for learning agents. Their measure judges safety from the agent's own state, such as its starting velocity, and from forward dynamics. Added to standard navigation rewards, it forms an intrinsic reward that favors both task success and collision avoidance. In numerical tests, collisions fall and robots keep more distance from obstacles. Travel time rises only slightly, they report.

Catalog record

CREATORGuihlerme Daubt 1 and Adrian Redder 2 1 Faculty of Technology, Bielefeld University, Germany 2 Department of Electrical Engineering and Information Technology, Paderborn University, Germany [email protected] , [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28805
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24241

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents Guihlerme Daubt 1 and Adrian Redder 2 1 Faculty of Technology, Bielefeld University, Germany 2 Department of Electrical Engineering and Information Technology, Paderborn University, Germany [email protected] , [email protected] Abstract Safe navigation in complex environments remains a central challenge for reinforcement learning (RL) in robotics. This paper introduces Continuous Space-Time Empowerment for Physics-informed (C-STEP) safe RL, a novel measure of agent-centric safety tailored to deterministic, continuous domains. This measure can be used to design physics-informed intrinsic rewards by augmenting positive navigation reward functions. The reward incorporates the agent’s internal states (e.g., initial velocity) and forward dynamics to differentiate safe from risky behavior. By integrating C-STEP with navigation rewards, we obtain an intrinsic reward function that jointly optimizes task completion and collision avoidance. Numerical results demonstrate fewer collisions, reduced proximity to obstacles, and only marginal increases in travel time. Overall, C-STEP offers an interpretable, physics-informed approach to reward shaping in RL, contributing to safety for agentic mobile robotic systems. 1 INTRODUCTION Agentic mobile robotic systems require safe and robust control in dynamic environments to ensure reliable navigation. Advances in reinforcement learning (RL) have shown promise for developing such highly adaptive policies. Previous works have explored safe RL methods by using control barrier functions [ 6 , 4 ] , robustness prioritization [ 15 ] , and risk-avoidance [ 2 ] . Nevertheless, safety remains a challenge when dealing with continuous space-time dynamics and unknown environmental conditions. Recent surveys have highlighted the potential of concepts from information theory for safe RL [ 8 , 1 ] . Considering this, we introduce a novel formulation of empowerment [ 11 ] for dynamical systems operating in continuous space and time. Empowerment is a measure of intrinsic agent control capacity in an environment and has traditionally been used in discrete-time stochastic computer networks. Herein, we propose a new definition of empowerment tailored for dynamical systems. By approximating empowerment through trajectory sampling and forward system dynamics, our approach captures the complex interaction between control strategies and safety, yielding an interpretable safety metric. With the empowerment approximation, we then formulate an intrinsic safe RL reward design. Our core contribution is a physics-informed intrinsic reward design that incorporates internal system states (such as velocity) to assess safety in navigation via the local empowerment approximation of a system. This empowerment approach can be integrated with positive navigation reward functions, serving as a complementary enhancement to promote physics-informed safety rather than replacing conventional reward designs. The rest of this paper is structured as follows. Section 2 provides background material. Section 3 discusses prior research. Section 4 introduces our methodology, starting with a new definition of empowerment for deterministic systems and culminating in our proposed C-STEP framework. Section 5 presents numerical experiments highlighting the safety benefits of our approach. Finally, we discuss C-STEP in comparison to other safe RL paradigms in Section 6 and conclude with remarks on future work in Section 7 . 2 Background We now recall background on information theory and RL. For in-depth background, see [ 3 ] and [ 20 ] , respectively. 2.1 Information Theory The differential entropy of a continuous random variable Y with probability density function f(y) and support set \mathcal{Y}\subset\mathbb{R}^{n} is defined as: h(Y)\coloneqq-\int_{\mathcal{Y}}f(y)\log\left(f(y)\right)\mathrm{d}y. (1) An important property is that differential entropy is maximized by a uniform distribution [ 3 , Chap. 12] , i.e., for any Y with compact support \mathcal{Y} : \max\limits_{p(y)}h(Y)=\log(\lambda(\mathcal{Y})), (2) where \lambda(\cdot) is the n -dimensional volume (Lebesgue measure). The conditional differential entropy h(Y\mid X) of random variables X,Y with probability density function g(x) and conditional probability density h(y\mid x) is defined as: h(Y\mid X)\coloneqq-\int_{\mathcal{X}}g(x)\int_{\mathcal{Y}}h(y\mid x)\log(h(y\mid x))\mathrm{d}y\mathrm{~d}x. (3) The mutual information is then defined as I(Y;X)\coloneqq h(Y)-h(Y\mid X) . Finally, the channel capacity from an “input” random variable X to an “output” variable Y is the maximal possible mutual information: \mathcal{C}(X\to Y)\coloneqq\max\limits_{p(x)}I(Y;X). (4) In the following sections, we adapt these core concepts of channel capacity and entropy to formulate a novel, agent-centric safety measure for deterministic robotic systems. Our central observation is that the channel capacity \mathcal{C}(X\to Y) can quantify safety. A system state is considered safe if the channel capacity from system control inputs to successive system states is sufficiently large. For deterministic systems, this capacity will be related to the logarithmic volume of reachable sets over a finite horizon. 2.2 Reinforcement Learning In RL, an agent learns to act in an environment defined by a Markov Decision Process (MDP) with state space \mathcal{S} , action space \mathcal{A} , transition kernel p , and reward function r:\mathcal{S}\times\mathcal{A}\to\mathbb{R} . At each state s\in\mathcal{S} , the agent picks an action a\in\mathcal{A} , receives a reward r(s,a) , and transitions to a successor state s^{\prime} with distribution p(\cdot\mid s,a) . The objective is to find a policy \pi:\mathcal{S}\rightarrow\mathcal{A} that maximizes the discounted return R_{0}\coloneqq\sum_{k=0}^{\infty}\gamma^{k}r(s_{k},a_{k}) . In model-free Deep RL, a policy \pi_{\theta} is parameterized by a deep neural network and improved iteratively using interaction data (s,a,r,s^{\prime}) to approximate a policy gradient. 3 Related Work Several studies have incorporated information-theoretic measures to promote exploration and skill acquisition in RL. In [ 14 ] , a variational lower bound of empowerment is used as an intrinsic reward for self-motivated learning. Building on this, [ 12 ] introduced a hierarchical empowerment framework for skill learning. Other works have applied empowerment to enhance self-optimizing networks [ 10 ] and to incentivize exploration in sparse-reward scenarios [ 13 , 17 ] . These methods typically approximate empowerment for discrete-time agents, often at high computational cost. Crucially, our work appears to be the first to leverage readily available forward dynamics of continuous-time mobile systems to approximate empowerment. Another research direction integrates physics-based insights into reward design. For instance, [ 9 ] used principles like energy conservation as intrinsic rewards. Other methods use reward functions incorporating model dynamics to facilitate diverse motion learning in complex agents [ 5 ] or evaluate performance based on velocity relative to system dynamics and a target [ 21 ] . In contrast, our method evaluates safety by leveraging the agent’s internal states and sampled forward dynamics to approximate reachable sets. We then introduce a multiplicative empowerment term applicable to any positive reward function. This method can be applied on top of reward designs and in conjunction with various safe RL methods [ 7 ] . This has not been explored in prior work, which used additive penalties to promote safety. 4 C-STEP: Continuous Space-Time Empowerment This section presents our core contribution, the Continuous Space-Time Empowerment for Physics-informed (C-STEP) safe RL framework. We begin by illustrating why the classical definition of empowerment is ill-suited for deterministic systems and propose a re-formulation. We then develop an intrinsic, physics-informed reward for safe navigation and provide a practical sampling-based method for its approximation. 4.1 Rethinking Empowerment for Deterministic Systems Empowerment, first introduced in [ 11 ] , measures the control an agent has over its future sensor states. For a discrete-time perception-action loop (PAL), where at every time k an agent picks an action U_{k} based on sensor state X_{k} , leading to a subsequent state X_{k+1} , empowerment is defined as the conditional channel capacity from actions to subsequent states. Definition 1

Read the complete text