Uncertainty-Resilient Active Intention Recognition for Robotic Assistants

Juan Carlos Saborío, Marc Vinci, Oscar Lima, Sebastian Stock, Lennart Niecksch, Martin Günther, Alexander Sung, Joachim Hertzberg, Martin Atzmueller · 2025

Read Uncertainty-Resilient Active Intention Recognition for Robotic Assistants by Juan Carlos Saborío, Marc Vinci, Oscar Lima, Sebastian Stock, Lennart Niecksch, Martin Günther, Alexander Sung, Joachim Hertzberg, Martin Atzmueller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19150.txt.

Introduction

Juan Carlos Saborío and a large team of co-authors published this robotics paper in 2025. A helpful robot assistant must guess what its human partner intends to do, then act before being told. Many systems dodge the hard part by waiting for explicit commands or by assuming near-perfect information. This paper targets the messy middle: intentions are uncertain, sensors are noisy, and plans can go wrong.

The team built a framework that reasons openly about that uncertainty. Its core is a planning model that weighs possible intentions and outcomes while new sensor data streams in, and it cooperates with the human rather than merely reacting. Planners for action are woven around that core. The system was tested on a physical robot, with results the authors call promising. They present the work as a step toward assistants that stay useful when perception fails.

Catalog record

CREATORJuan Carlos Saborío, Marc Vinci, Oscar Lima, Sebastian Stock, Lennart Niecksch, Martin Günther, Alexander Sung, Joachim Hertzberg, Martin Atzmueller
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19150

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Uncertainty-Resilient Active Intention Recognition for Robotic Assistants Juan Carlos Saborío, Marc Vinci, Oscar Lima, Sebastian Stock, Lennart Niecksch, Martin Günther, Alexander Sung, Joachim Hertzberg, Martin Atzmueller Abstract Purposeful behavior in robotic assistants requires the integration of multiple components and technological advances. Often, the problem is reduced to recognizing explicit prompts, which limits autonomy, or is oversimplified through assumptions such as near-perfect information. We argue that a critical gap remains unaddressed – specifically, the challenge of reasoning about the uncertain outcomes and perception errors inherent to human intention recognition. In response, we present a framework designed to be resilient to uncertainty and sensor noise, integrating real-time sensor data with a combination of planners. Centered around an intention-recognition POMDP, our approach addresses cooperative planning and acting under uncertainty. Our integrated framework has been successfully tested on a physical robot with promising results. I Introduction Robotic assistants may be integrated into modern industrial environments, e.g., delivering tools, parts or modules interleaved with tidying the workspace. Such tasks, however, require a combination of robust planning, navigation, grasping, and perception–particularly when explicit commands are not available and the robot must identify and pursue goals, in collaborative spaces shared with people. Existing work already addresses several building blocks necessary for such behavior, ranging from flexible robot control [ 1 , 2 ] to intention recognition (IR) and partially observable planning [ 3 , 4 , 5 , 6 ] . IR based on Partially Observable Markov Decision Processes (POMDPs) provides a formal framework for handling uncertainty and information gathering, but a gap remains when transfering these methods to real robotic platforms. Recent approaches include detecting motion direction [ 1 ] or recognizing explicit gestures that trigger replanning [ 2 ] , but both are limited to activities rather than predicting future needs, failing to address the inherent uncertainty in IR which stems from both perception errors and the unpredictability of actions with delayed outcomes. Motivated by a human-robot collaboration scenario, we propose an integrated framework that tackles several open challenges in automated robotic assistance, with a focus on uncertainty and IR . Key challenges include dynamic environments, incomplete information, and the need to leverage real-time sensor data for online action selection. The collaborative aspect is addressed through human IR : the ability of an agent to infer and predict another’s upcoming actions or needs. This interpretation assumes agents behave in a goal-directed manner, where an intention refers to a potential (sub)goal or future action within their plan [ 7 ] , rather than a currently ongoing activity. Acting based on predicted intentions enables autonomy that goes beyond avoiding collisions or reacting to explicit instructions, such as spoken commands or gestures. We implemented and tested our approach through integrated system comprising external cameras, a mobile robot, and multiple software modules for data acquisition, processing, task selection, and action execution. At its core is an active goal recognition (AGR) POMDP model and planner, which integrates external information and selects the next best action, corresponding to a high-level task further decomposed and executed onboard the robot by a hierarchy of planners. Each module was validated independently, and the integrated framework was tested on a physical robot in a simplified industrial assembly scenario. In our tests, a human worker assembles one of two types of insect hotels , small wooden structures designed to shelter insects (see Fig. 1 ). Each hotel type requires both common and type-specific parts, all color-coded, that may be assembled in different orders, generalizing the six-part ball point pen used in [ 5 ] . The robot supports the worker by monitoring part usage and availability and by delivering missing parts as needed . Our system was able to assist a human worker effectively despite uncertainty and sensor noise and without relying on explicit instructions. Fig. 1: A ten-part insect hotel: individual parts (left) and assembled (right). The main contribution of this paper is the integrated framework for uncertainty-resilient active IR in robotic assistants, focusing on information and outcome uncertainty in cooperative settings. We start by reviewing related work in IR with robotic applications, followed by a detailed description of our framework, evaluated on two sets of simulated experiments. We conclude with a brief discussion of current limitations and directions for future work. II Related Work Intention Recognition in HRC. Recent advances in IR for human-robot collaboration (HRC) adopt a reactive rather than proactive perspective. Examples include motion and gesture recognition to trigger HTN replanning [ 2 ] , and an RNN approach to detect motion and trajectories to avoid interrupting or disrupting an ongoing task [ 1 ] . We address this gap in proactive assistance by recognizing needs or goals, and not merely ongoing actions. Another approach models human actions as an “intention graph” and uses an offline POMDP planner for IR and action selection [ 5 ] , but assumes the robot performs the same actions as a human, on the same object by taking turns with no synchronization or coordination steps. Furthermore no robots were actually involved and there is no mention of physical limitations, failures or delays. We use our own extended POMDP formulation [ 6 ] , that allows a physical robot to reason about its own actions and their effect on a (human) worker, coordinating their behavior in a shared cooperative space. The resulting large POMDP is solved online, without prior factorization, by integrating noisy sensor information into an optimization process supporting joint planning and execution. From Perception to Intention. Recent approaches in HRC have achieved strong performance in activity recognition, based on gestures and ongoing tasks using skeleton-based tracking [ 8 ] , hand-object contact modeling [ 9 ] , or HMMs [ 10 ] . We instead focus on generating high-level abstractions (e.g., state and subgoal conditions) from multimodal sensor data, that become informative observations for the POMDP planner. In our demo scenario, we combined RGB-D cameras with a trained YOLOv8 [ 11 ] to detect relevant parts and track task progress, but any setup is feasible as long as similar abstractions can be generated. Intention, Plan and Goal Recognition. IR is traditionally restricted to observing and classifying the behavior of an agent [ 12 , 3 , 13 , 14 ] , relying on plan libraries with precomputed plans or a grammar to construct them [ 15 , 13 , 16 ] or through reverse planning methods that expand the space of plans and approximate possible goals [ 12 , 7 ] , also covering the cases of missing information [ 17 ] and partially observable domains [ 3 ] . We build upon these advances with a focus on planning and acting online , important in large and complex domains [ 18 ] , and also by adopting the active approach [ 4 ] , where the robot is not just a passive observer but a participant with its own actions. Furthermore, we don’t rely on state factoring or other types of preprocessing to simplify the domain [ 6 ] . POMDP Planning in IR . POMDP-based IR approaches in robotics often rely on offline, point-based solvers [ 5 , 18 ] , limited to small problems and factored state representations. We instead use RAGE,

Read the complete text