Situationally-Aware Dynamics Learning
Alejandro Murillo-González and Lantao Liu · 2025
Read Situationally-Aware Dynamics Learning by Alejandro Murillo-González and Lantao Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19574.txt.
Introduction
Alejandro Murillo-González and Lantao Liu propose in 2025 a way for robots to learn how their world behaves while staying aware of hidden situations. A robot in a messy setting cannot see every factor that shapes its motion and rewards, so its model goes stale when conditions quietly shift. Their framework names those hidden drivers as parameters inside the decision process itself.
The robot learns, online, the joint pattern of its recent state changes, which acts as a rich signal of both its own state and outside factors. A Bayesian change spotting method, extended to many variables, cuts the stream where the underlying process seems to switch. The current segment is turned into a symbolic label for the situation, and the motion model is updated with that label attached. Tests in a hard real world task back the claim of safer, more adaptive behavior.
Catalog record
| CREATOR | Alejandro Murillo-González and Lantao Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 107 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19574
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Situationally-Aware Dynamics Learning Alejandro Murillo-González and Lantao Liu Abstract Autonomous robots operating in complex, unstructured environments face significant challenges due to latent, unobserved factors that obscure their understanding of both their internal state and the external world. Addressing this challenge would enable robots to develop a more profound grasp of their operational context. To tackle this, we propose a novel framework for online learning of hidden state representations, with which the robots can adapt in real-time to uncertain and dynamic conditions that would otherwise be ambiguous and result in suboptimal or erroneous behaviors. Our approach is formalized as a Generalized Hidden Parameter Markov Decision Process, which explicitly models the influence of unobserved parameters on both transition dynamics and reward structures. Our core innovation lies in learning online the joint distribution of state transitions, which serves as an expressive representation of latent ego- and environmental-factors. This probabilistic approach supports the identification and adaptation to different operational situations, improving robustness and safety. Through a multivariate extension of Bayesian Online Changepoint Detection, our method segments changes in the underlying data generating process governing the robot’s dynamics. The robot’s transition model is then informed with a symbolic representation of the current situation derived from the joint distribution of latest state transitions, enabling adaptive and context-aware decision-making. To showcase the real-world effectiveness, we validate our approach in the challenging task of unstructured terrain navigation, where unmodeled and unmeasured terrain characteristics can significantly impact the robot’s motion. Extensive experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies. Accompanying video: https://youtu.be/VKR18WaSCAk . I Introduction Figure 1 : Motivating Example. When the robot has access only to incomplete state information, the same action, \mathbf{a} , can yield significantly different outcomes. In this example, unmodeled terrain types—factors absent from the state and transition model—affect the robot’s response, resulting in distinct changes in position and heading ( \Delta x , \Delta y , and \Delta\psi , respectively) for the same control action (commanded linear and angular velocity). Generalized Hidden Parameter Markov Decision Processes (GHP-MDPs) address this challenge by modeling the unknown latent factors responsible for these discrepancies and incorporating them into the transition model, thereby enhancing predictive accuracy. (Illustrative environment images generated with Gemini AI.) For reliable robot deployment in unstructured and dynamic environments, it is crucial for robotic systems to adapt to unforeseen challenges and operate effectively under uncertainty. Consider, for example, an Unmanned Ground Vehicle (UGV) navigating steep and rugged terrain. Such a robot may become immobilized due to unexpected factors, ranging from unmodeled terrain properties to variations in its own internal dynamics. These factors are inherently difficult, if not impossible, to anticipate exhaustively [ 23 , 53 , 32 , 84 , 99 , 97 ] . This challenge is further exacerbated by the impracticality of equipping robots with every conceivable sensor and analysis algorithm needed to fully capture the world’s and robot’s full state in all scenarios. Consequently, robots must learn to operate effectively using only partial information derived from their observable state—the limited set of measurements and estimates available in real time. This necessitates the development of adaptive mechanisms capable of inferring and representing the hidden aspects of the robot and its environment to inform decision-making and enhance resilience in uncertain conditions [ 65 , 26 , 48 , 45 ] . Furthermore, when a robot relies on partial information to operate on a complex environment, the risk of ambiguous scenario representation becomes significant [ 66 , 73 , 12 ] . Such ambiguity is undesirable, as it increases uncertainty during decision-making, often leading to suboptimal plans and, in some cases, unsafe or hazardous behaviors. This challenge is exemplified in Figure 1 : a UGV with incomplete state information—lacking a (robust) scene interpretation module to detect the terrain type it is traversing—executes an action from what appears to be the same state. However, the outcomes of these actions differ drastically, revealing a discrepancy that cannot be attributed solely to aleatoric errors (those arising from process or observation noise). This highlights the critical need for robust methods to infer and resolve hidden state ambiguities to ensure reliable robot operation. To address this, we propose a framework for learning online a representation of the unmodeled and unobserved latent factors that cause such ambiguities. This represents a significant challenge, as it requires balancing the tradeoff between representation capacity, data- and learning-efficiency, and minimization of inductive biases that could inadvertently exclude critical latent factors [ 57 , 39 ] . Considering this, we introduce an online hidden state representation learning method that enables the robot to recover and leverage representations of latent world and robot factors in real time. By utilizing this learned representation, the robot can dynamically adapt its behavior and reliably accomplish its tasks, even in uncertain and unstructured environments. Recent studies have introduced various methods that leverage learned auxiliary representations during dynamics or policy learning to enable robot adaptation [ 58 , 50 , 51 , 74 , 61 ] . However, these methods are data-intensive and often rely on multi-stage procedures that require privileged information, such as ground-truth data that is difficult to obtain or is only available in simulation. To overcome these limitations, our proposed method learns such representations in an online and unsupervised manner, thus eliminating the need for multi-stage approaches. We achieved this by formulating the problem within the framework of Generalized Hidden Parameter Markov Decision Processes (GHP-MDPs) [ 70 , 22 ] . An important component of our framework is to represent the hidden state by learning online the joint distribution of the state transitions . We refer to this state transition distribution at a given moment as the situation of the robot. This approach enables the robot to capture an expressive probabilistic representation of its current context, encompassing both its state and the underlying environmental conditions. By leveraging proprioceptive sensing, such as inertial measurements, to assess the robot’s state, this representation becomes sufficiently general to effectively disambiguate the robot’s state across various scenarios. Additionally, as illustrated in Figure 1 , the state evolution exhibits diverse probability density profiles under different conditions. This diversity facilitates the identification of latent factors that govern distinct scenarios. In this way, the situation can be understood as the Underlying Data Generating Process (UDGP)—a conceptual model that characterizes the true, often hidden, factors governing the system’s dynamics at any given moment [ 30 , p. 109] . The UDGP encapsulates the combined influence of both robot-specific and environmental variables that affect the robot’s behavior, enabling a more accurate understanding of its state. To obtain this situation distribution in real-time as the robot operates, we introduce a multivariate extension of Bayesian Online Changepoint Detection (BOCD) [ 3 ] . This extension allows us to model the situation the robot is currently operating under and detect when a situation change occurs. Concretely, in this paper we make the following contributions: • We introduce an efficient method that enables a robot to adapt online to the latent factors influencing its performance. We present a multivariate BOCD approach to estimate the distribution of the transition dynamics in real time. We show that these distributions serve as