Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability
Yushen Liu, Yin-Jen Chen, Ziyi Chen, Tao Wang, Heng Huang, Xugui Zhou, Yanfu Zhang · 2026
Read Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability by Yushen Liu, Yin-Jen Chen, Ziyi Chen, Tao Wang, Heng Huang, Xugui Zhou, Yanfu Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14246.txt.
Introduction
Liu and six co-authors propose a light risk-gated reinforcement learning approximation for risk-aware control with partial views. Safety-critical control uses risk-aware Markov decision processes with partial views. The method builds a finite-history proxy state and learns a forecaster of near-term safety breaches per candidate action. That predicted risk serves as a penalty in value learning and as a gate between optimistic and conservative value estimates. In adult and teen glucose cohorts, the method improves glycemic tradeoffs and cuts runtime against a belief baseline.
Catalog record
| CREATOR | Yushen Liu, Yin-Jen Chen, Ziyi Chen, Tao Wang, Heng Huang, Xugui Zhou, Yanfu Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79704 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14246
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability Yushen Liu, Yin-Jen Chen, Ziyi Chen, Tao Wang, Heng Huang, Xugui Zhou, Yanfu Zhang Abstract Many safety-critical control problems are modeled as risk-sensitive partially observable Markov decision processes, where the controller must make decisions from incomplete observations while balancing task performance against safety risk. Although belief-space planning provides a principled solution, maintaining and planning over beliefs can be computationally costly and sensitive to model specification in practical domains. We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability. The method constructs a compact finite-history proxy state and learns an action-conditioned predictor of near-term safety violation. This predicted candidate-action risk is used in two complementary ways: as a risk penalty during value learning, and as a decision-time gate that interpolates between optimistic and conservative ensemble value estimates. As a result, low-risk actions are evaluated closer to reward-seeking estimates, while high-risk actions are evaluated more conservatively. We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation. Across adult and adolescent glucose-control cohorts, the method improves overall glycemic tradeoffs and substantially reduces runtime relative to a belief-space planning baseline. On Safety-Gym navigation benchmarks, it achieves a more favorable reward-cost balance than unconstrained RL and several standard safe-RL baselines. These results suggest that action-conditioned near-term risk can provide an effective local signal for approximate risk-sensitive POMDP control when full belief-space planning is impractical. 1 Introduction Many real-world control problems are both safety-critical and partially observable, arising in settings where agents must act under noisy sensing, imperfect control, and uncertain environment dynamics 17 ; 5 . In such settings, the agent must choose actions from incomplete observations while avoiding rare but costly failures. Examples include automated glucose regulation, where hidden meal absorption and delayed insulin dynamics shape future risk, and safety-constrained navigation, where latent disturbances or limited observability can turn seemingly good actions into unsafe ones. In both cases, high performance alone is insufficient: the controller must act conservatively when near-term hazard is elevated, yet remain effective enough to achieve the task. A principled framework for such problems is the partially observable Markov decision process (POMDP) 6 , in which the agent maintains a belief over latent states and plans in belief space. In practice, however, belief-state control can be difficult to deploy in safety-critical settings 26 . Accurate belief updates require transition and observation models that are often unavailable, and online belief-space planning can remain computationally expensive even with approximate models 27 . Moreover, while a full belief state is the principled sufficient statistic for optimal POMDP control, it may be more information than is needed by a particular approximate controller in some safety-critical tasks. For the settings considered here, we focus on a narrower decision question: whether a candidate action is likely to cause a near-term safety violation. We therefore investigate whether an action-conditioned estimate of near-term hazard, computed from recent history, can provide enough safety-relevant information for effective control without explicitly maintaining a full belief state 13 ; 36 ; 5 . This suggests that, for controllers whose safety decisions are dominated by short-horizon constraint risk, estimating action-conditioned near-term hazard may provide a practical alternative to maintaining a full latent-state belief. This observation motivates a lighter alternative to explicit belief-space control. We propose a risk-gated reinforcement learning framework for safety-critical control under partial observability that avoids explicit belief updates by using a compact history-dependent proxy state and an action-conditioned estimate of near-term hazard. Figure 1 summarizes how the proposed approach is positioned relative to classical POMDP planning and standard safe RL: it targets partial observability like POMDP methods, but avoids explicit belief updates, while retaining the computational simplicity of RL-style action evaluation. Positioning. Our method targets safety-critical control under partial observability, such as automated glucose regulation and safety-constrained navigation. Unlike classical POMDP methods, it avoids explicit belief updates and online belief-space planning by using a compact proxy state. Unlike standard safe RL, which often assumes full state access, it estimates action-conditioned near-term risk from partial observations and uses this risk to interpolate between optimistic and conservative ensemble Q-values. This provides a lightweight middle ground between belief-space POMDP planning and fully observable safe RL. Figure 1 : Positioning of risk-gated RL relative to classical POMDPs and standard safe RL. The key design choice is to estimate near-term risk for each candidate action directly from recent history, rather than infer a full latent-state posterior 23 ; 22 . For each candidate action a\in\mathcal{A} , the controller predicts \hat{\rho}_{t}(a)\approx\mathbb{P}(E_{t}(a)=1\mid H_{t}), where E_{t}(a) denotes a safety violation within a short horizon. This predicted hazard is then used to gate action evaluation between more optimistic and more conservative value estimates, so that the controller behaves more aggressively for low-risk actions and more conservatively for high-risk ones. Our approach can be interpreted as a decision-oriented compressed-POMDP approximation. Instead of learning or updating a full latent belief, we retain two quantities aimed at action selection: a compressed summary of recent observations and an action-conditioned predictor of near-term hazard. This avoids explicit belief updates while aiming to retain the information most relevant to short-horizon safety-aware control. The resulting framework is lightweight, modular, and easy to instantiate in practical discretized action spaces. Additional intuition on the relation between the proposed method, latent-state POMDP views, and a surrogate risk-aware representation is provided in Appendix A . We evaluate the proposed method in two different safety-critical partially observable settings: automated glucose regulation in the FDA-accepted UVa/Padova simulator 18 ; 39 , and safe navigation in Safety-Gym benchmarks 11 . These domains differ substantially in dynamics, observations, and task objectives, but share the same central challenge: the agent must act under partial observability while balancing performance and constraint avoidance. Across both domains, we find that risk-gated control yields a strong balance among performance, safety, and computational efficiency relative to both belief-based planning and standard safe-RL baselines. This paper makes three main contributions. First, we introduce a framework for safety-critical partially observable control that avoids explicit belief updates by using a compact history-dependent proxy state and an action-conditioned estimate of near-term hazard. Second, we propose a risk-gated action-evaluation rule that uses predicted hazard to interpolate between optimistic and conservative value estimates. Third, we demonstrate across glucose regulation and Safety-Gym navigation that this design yields a favorable balance among task performance, safety, and computational efficiency. 2 Related Work Partial observability and belief-based control. POMDPs provide a principled framework for sequential decision-making under partial observability by maintaining a belief state over latent variables 6 . In principle, belief-space planning can recover optimal decisions under uncertainty, but in practice exact filtering and planning are often intractable in continuous, high-dimensional, or safety-critical settings 19 ; 26 . Approximate belief compression, learned latent-state models, and recurrent RL reduce this burden, but they still aim to preserve a broad summary of future-relevant uncertainty through either explicit belief maintenance or implicit hidden state 16 ; 10 . These approaches aim to preserve information useful for general future prediction or long-horizon control. Our work takes a more decision-oriented perspective. Rather than learning or updating a general-purpose belief or latent state, we focus on a narrower safety-relevant quantity: the action-conditioned probability of a near-term violation.