Bresa : B io-inspired Re flexive Sa fe Reinforcement Learning for Contact-Rich Robotic Tasks

Heng Zhang, Gokhan Solak, Arash Ajoudani · 2025

Read Bresa : B io-inspired Re flexive Sa fe Reinforcement Learning for Contact-Rich Robotic Tasks by Heng Zhang, Gokhan Solak, Arash Ajoudani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21989.txt.

Introduction

Heng Zhang, Gokhan Solak, and Arash Ajoudani present a safety method for robots that learn tasks with physical contact, in a 2025 paper. Contact-rich work in messy settings is risky. Most safe learning methods guard exploration or add recovery moves. They often miss low-level safety, where fast reflexes matter most.

Their method, Bresa, takes its cue from animal reflexes. Task learning and safety learning run apart. A safety critic scores each action's risk and runs faster than the task solver. A second layer adjusts how stiffly the robot pushes back. By splitting the jobs, the robot can keep learning the task while a reflex layer guards each motion as it happens.

Catalog record

CREATORHeng Zhang, Gokhan Solak, Arash Ajoudani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21989

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bresa : B io-inspired Re flexive Sa fe Reinforcement Learning for Contact-Rich Robotic Tasks Heng Zhang, Gokhan Solak, Arash Ajoudani Abstract Ensuring safety in reinforcement learning (RL)-based robotic systems is a critical challenge, especially in contact-rich tasks within unstructured environments. While the state-of-the-art safe RL approaches mitigate risks through safe exploration or high-level recovery mechanisms, they often overlook low-level execution safety, where reflexive responses to potential hazards are crucial. Similarly, variable impedance control (VIC) enhances safety by adjusting the robot’s mechanical response , yet lacks a systematic way to adapt parameters, such as stiffness and damping throughout the task. In this paper, we propose Bresa, a B io-inspired Re flexive Hierarchical Sa fe RL method inspired by biological reflexes. Our method decouples task learning from safety learning, incorporating a safety critic network that evaluates action risks and operates at a higher frequency than the task solver. Unlike existing recovery-based methods, our safety critic functions at a low-level control layer, allowing real-time intervention when unsafe conditions arise. The task-solving RL policy, running at a lower frequency, focuses on high-level planning (decision-making), while the safety critic ensures instantaneous safety corrections. We validate Bresa on multiple tasks including a contact-rich robotic task, demonstrating its reflexive ability to enhance safety, and adaptability in unforeseen dynamic environments. Our results show that Bresa outperforms the baseline, providing a robust and reflexive safety mechanism that bridges the gap between high-level planning and low-level execution. Real-world experiments and supplementary material are available at project website https://jack-sherman01.github.io/Bresa . I INTRODUCTION (a) (b) Fig. 1 : a) Bresa framework. The RL agent operates at the decision loop , planning the high-level action \mathbf{a} that is executed by the trajectory controller. The controller operates at a high-frequency control loop , executing the low-level action \hat{\mathbf{a}} based on the state feedback \hat{\mathbf{s}} at each control step. The reflex mechanism gives the system a quick reaction capability by interrupting the control loop in the case of high risk. b) A simplified illustration of the human central nervous system. While high-level decisions are made in the brain, safety-related reflexes are managed by the spinal cord, allowing for faster responses that override slower, more complex decision-making processes. Robotic actions in the real world present two major challenges: the complexity of unstructured environments and the safety hazards associated with physical interactions [ 1 ] . RL-based robotic systems have the potential to address both challenges to enable effective automated learning and exploration in such environments [ 2 ] . Traditionally, the complexity challenge has received significant attention, while the safety challenge has gained focus more recently, especially in contact-rich tasks [ 1 ] . Drawing inspiration from the animal kingdom, where evolutionary processes have led to the development of solutions to these challenges, we propose a novel approach. Specifically, this paper draws on the reflex mechanisms inherent to vertebrates to enhance the safety and robustness of RL systems. The complexity challenge can be mitigated by imposing a hierarchy between long-term task-level actions and short-term motor-level actions. Early RL studies have shown the advantage of decomposing complex tasks into smaller subtasks [ 3 ] . Furthermore, it was recently shown that considering actions in task-space leads to more efficient learning, in comparison to using joint-space actions [ 4 ] . Thus, we assume a hierarchy between the high-level decision-making and low-level control loops as shown in Fig. 1 .a. The safety challenge has gained more attention recently due to the black-box nature of traditional learning-based RL models in safety-critical environments, under the umbrella of safe RL [ 5 ] . Specifically, we follow the hierarchical safe RL methods, which answer the task-solving and safety problems through separate learned components. A crucial component is a safety critic network that estimates the risk of failure to limit access to risky states [ 6 ] . Furthermore, a separate recovery policy can be deployed in addition to the usual task policy for learning more complex risk-evasive behavior [ 7 ] . Integrating variable impedance control (VIC) into hierarchical safe RL is shown to enhance contact-rich interaction safety further through adaptable compliant behavior [ 8 ] . However, existing safety methods primarily function at the decision-making stage, often overlooking risks that emerge during low-level action execution. Addressing safety in the decision-making loop leaves the system vulnerable to many safety-critical events occurring at the control stage. An action may be initially evaluated as safe, however, the risk may increase during execution because of the dynamicity, stochasticity and partial observability of the environment [ 9 , 10 ] . Specifically, sensor noise, dynamical effects, or delays in actuation can lead to deviations from the intended trajectory, pushing the system into unsafe states. External factors such as changing environmental conditions or unforeseen obstacles can further compromise safety during execution. Furthermore, contact-rich tasks are particularly prone to control-time risks, as forming and breaking physical contact introduces highly non-linear and discontinuous dynamics, requiring similarly dynamic responses [ 11 , 12 ] . Therefore, integrating a dedicated low-level safety mechanism is essential for ensuring instantaneous corrective actions during execution. We propose the B iomimetic Re flexive Hierarchical Sa fe RL ( Bresa ) to address this gap. Our method is inspired by the reflex mechanism common to all vertebrates. As illustrated for humans in Fig. 1 .b, reflexes are managed by the spinal cord, bypassing the complex reasoning in the brain [ 13 ] . Similarly, we place the safety evaluation in the high-frequency control loop to immediately interrupt the execution when the risk exceeds a threshold, as shown in Fig. 1 .a. The reflex triggers the recovery policy that aims to escape the danger, instead of the task policy that aims to solve the task while it is safe. We evaluate our method on both a 2-dimensional navigation task and a contact-rich maze exploration task. For the maze exploration task we first train our method in Mujoco simulator, and then transfer to a real-world robot platform. We compare Bresa to the baseline hierarchical safe RL method [ 8 ] where the safety-checking and task-solving happens together in the decision loop . The results show that Bresa significantly decreases safety violations, and consequently improves efficiency in task learning. The novel method ensures a reflexive and adaptive response to potential hazards, significantly improving safety in contact-rich and uncertain environments. An important aspect of our method is that it is designed to work on unknown contact-rich environments. Bresa learns the safety notion from data and reactively avoids risks. Our framework allows contacts with the environment, that is fundamental in contact-rich tasks. In that regard, it differs from the works like [ 14 , 15 ] that model the safety constraints geometrically and project the RL action into a safe tangential space to avoid obstacles. Another related work [ 16 ] , uses an RL agent for planning and an MPC for safe low-level execution. This work also defines the constraints as contact avoidance. The MPC ensures safety at control-level, given the obstacle positions and a dynamics model. The RL-based contact-rich applications usually take advantage of adaptable impedance/admittance controllers [ 17 , 18 ] . These two works rely on simulators for safe training, and then transfer the learned model

Read the complete text