Hierarchical Intention-Aware Expressive Motion Generation for Humanoid Robots

Lingfan Bao, Yan Pan, Tianhu Peng, Dimitrios Kanoulas, Chengxu Zhou · 2025

Read Hierarchical Intention-Aware Expressive Motion Generation for Humanoid Robots by Lingfan Bao, Yan Pan, Tianhu Peng, Dimitrios Kanoulas, Chengxu Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01563.txt.

Introduction

Lingfan Bao and co-authors build a two-layer system that lets humanoid robots read a person's intent and answer with fitting gestures, in a 2025 paper. Robots that work near people need to move in ways that feel social and timely, but fixed gesture libraries feel stiff and heavy generative models run too slowly. Their design splits the task between a reasoning layer and a motion layer.

The top layer uses a language model with structured prompts, confidence scores, and fallback moves to refine what the person wants. The lower layer, a diffusion model working in a compact space, produces varied, plausible gestures in real time. Trials on a real robot showed the system held up in live, changing encounters.

Catalog record

CREATORLingfan Bao, Yan Pan, Tianhu Peng, Dimitrios Kanoulas, Chengxu Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01563

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hierarchical Intention-Aware Expressive Motion Generation for Humanoid Robots Lingfan Bao, Yan Pan, Tianhu Peng, Dimitrios Kanoulas, Chengxu Zhou Abstract Effective human-robot interaction requires robots to identify human intentions and generate expressive, socially appropriate motions in real-time. Existing approaches often rely on fixed motion libraries or computationally expensive generative models. We propose a hierarchical framework that combines intention-aware reasoning via in-context learning (ICL) with real-time motion generation using diffusion models. Our system introduces structured prompting with confidence scoring, fallback behaviors, and social context awareness to enable intention refinement and adaptive response. Leveraging large-scale motion datasets and efficient latent-space denoising, the framework generates diverse, physically plausible gestures suitable for dynamic humanoid interactions. Experimental validation on a physical platform demonstrates the robustness and social alignment of our method in realistic scenarios. I Introduction In human society, communication takes place through two primary channels: verbal and non-verbal. While verbal language conveys explicit information, non-verbal cues, such as facial expressions, body posture, and gestures, carry rich and socially signals that reflect intention, emotion, and context. These expressive cues play a crucial role in natural interaction and mutual understanding for human robot interaction [ 1 ] . Among various robotic embodiments, humanoid robots are uniquely suited to support non-verbal social communication [ 2 ] . Their anthropomorphic structure allows for human-like motion and gesture execution, making them ideal for conveying expressive intent and enhancing user engagement in human-robot interaction [ 3 , 4 ] . On one hand, current research in humanoid robotics has made significant progress in motion control, focusing primarily on locomotion [ 5 , 6 ] , and manipulation [ 7 ] . These approaches aim for physical robustness and task execution, but often overlook the role of expressive, socially meaningful motions. On the other hand, human-robot interaction (HRI) research emphasises communication and social engagement, frequently using language-based interfaces [ 8 , 4 ] or rule-based behavioural templates [ 9 , 10 ] . However, these systems often lack physical embodiment, and rarely adaptively express nuanced human-like gestures in response to dynamic interaction contexts. Despite progress in both domains, there remains a fundamental disconnect: few systems are capable of interpreting human intention and generating appropriate upper-body expressive gestures in real time. This limits the ability of humanoid robots to participate in socially natural, co-present interaction scenarios. To develop a flexible and scalable framework that allows humanoid robots to perceive social cues and autonomously generate natural in real-time interactions, we introduce a scalable, intention-aware framework HIAER for expressive humanoid responses that integrates a large vision-language model (VLM) with real time generative motion control, enabling social interactions grounded in perception, intent inference, and gesture execution. We firstly leverage the in-context learning [ 11 , 12 ] capability of VLMs to interpret human intention based on prompt and body language. The model generates structured gestural prompts that reflect the inferred social context. At the motion planning lay, we utilize a text-to-motion diffusion model [ 13 ] to produce temporally coherent and socially expressive motion sequences in real time, conditioned on the high-level prompts. Finally, the motion will be conducted through a low-level controller to execute the generated motion on a physical humanoid platform. Our system empowers humanoid robots to produce socially aligned, physically executable gestures in real time. The key contributions of this work are in conclusion: • We present a low-latency, end-to-end framework that is validated through physical execution on a humanoid platform, demonstrating its effectiveness in real-world human-robot interaction tasks. • We introduce a iterative intention refinement, allowing the robot to adjust its response based on real-time feedback from human behaviour and history information. • We introduce a fallback strategy that generates cautious and socially appropriate gestures under ambiguous or low-confidence predictions. In our experiments, we evaluate the proposed framework across five representative categories of real-world human-robot interactions. The system achieves execution success in real robot, and demonstrates strong intention refinement capabilities, robust fallback behavior under ambiguous inputs, and context-sensitive gesture selection grounded in social cues. Fig. 1 : Overall framework of the proposed work. The system receives multi-modal user inputs, vision, language, and I/O history, and fuses them for gesture interpretation. A frozen VLM , guided by a pre-defined prompt and few-shot examples , infers the user’s intent and selects a context-appropriate gesture. If confidence is low, a fallback policy is triggered. The selected action is sent to a real-time motion generation module DART and retargeted for execution on a physical robot. Pink represents input modalities; Blue represents intention-aware reasoning; Green represents motion synthesis and execution. Dashed arrows denote initial one-time inputs, and solid arrows indicate inputs continuously provided throughout interaction. II Related Work II-A Expressive Human-Robot Interactions Non-verbal communication plays a critical role in human interactions, conveying emotions, intentions, and social cues beyond spoken language. Recent studies have shown that when interacting with AI agents, users expect not only coherent verbal responses but also appropriate gestures and postures that align with the conversational context [ 4 ] . In addition, previous works have demonstrated that enabling robots to generate expressive motions significantly enhances communication quality and user engagement during human-robot interaction [ 14 ] . While recent frameworks have enabled humanoid robots to execute diverse and expressive motions, they often lack the ability to engage in intention-aware interactions with humans. For example, ExBody [ 15 ] enables humanoid robots to learn upper-body expressive motions through imitation learning from large-scale human motion datasets and generative models. Similarly, OmniH2O [ 16 ] demonstrates a system capable of producing a wide range of whole-body motions using data-driven techniques. However, these works primarily focus on motion execution itself, without considering how to select or adapt motions in response to dynamic human social cues. Traditional methods for expressive human-robot interaction behaviors generally fall into two categories: rule-based and template-based. Rule-based methods rely on predefined rules, offering predictability but limited adaptability to dynamic social contexts [ 10 , 9 , 17 ] . Template-based methods use motion templates from expert demonstrations to increase flexibility but still impose rigid structural constraints, limiting diverse and context-sensitive behavior generation. The rapid advancement of large language models and VLMs has significantly enhanced robots’ ability to understand and reason about human intentions across diverse and dynamic scenarios. In the field of legged robotics, recent works have primarily leveraged LLMs and VLMs to solve task-oriented problems such as locomotion planning and control [ 18 , 19 ] . In contrast, our work employs ICL with few-shot examples to teach the agent to reason about human intentions and respond through corresponding expressive gestures. Rather than focusing solely on task execution, we utilize LLMs to infer human social cues and generate appropriate upper-body actions for humanoid robots in real-time interactions. II-B Expressive Motion Generation Generating expressive and natural motions is critical for effective human-robot interaction, as it enables robots to clearly convey social intent and respond dynamically in real-world contexts. Current data-driven approaches broadly include retrieval-based methods and generative models. Retrieval methods select motion sequences from existing databases, offering realism and quick inference but limited flexibility due to library constraints. Generative models, such as GANs [ 20 ] , VAEs [ 21 ] , and diffusion models [ 22 ] , allow for novel and contextually

Read the complete text