WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots

Yuxuan Hu, Kuangji Zuo, Boyu Ma, Shihao Li, Zhaoyang Xia · 2026

Read WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots by Yuxuan Hu, Kuangji Zuo, Boyu Ma, Shihao Li, Zhaoyang Xia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07454.txt.

Introduction

Yuxuan Hu and colleagues introduce WaveMan, a sensing system for humanoid robots, in this 2026 preprint. Their goal is room-scale human interaction perception inside homes. Two demands shape the work: robots must cope with users standing anywhere, and they must respect user privacy. Cameras can fail the second demand, so the team turns to radio sensing instead.

WaveMan uses millimeter-wave signals, which can sense people without recording images. Earlier mmWave systems, the authors note, generalize poorly to new distances or viewpoints. A system trained in one spot may misread a person in another. WaveMan is built to handle that spatial variation across a whole room. The paper matters for home robotics, where privacy and reliability must come together.

Catalog record

CREATORYuxuan Hu, Kuangji Zuo, Boyu Ma, Shihao Li, Zhaoyang Xia
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-07454

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots Thanks: This work is supported by MOE Singapore Tier 1 Grant RG83/25, RS36/24 and a Start-up Grant from Nanyang Technological University. (Corresponding authors: Feng Xu, Jianfei Yang). Thanks: Yuxuan Hu, Zhaoyang Xia, and Feng Xu are with the Key Laboratory for Information Science of Electromagnetic Waves, Ministry of Education, School of Information Science and Technology, Fudan University, Shanghai 200433, China (e-mail: [email protected]; [email protected]; [email protected]). Thanks: Yuxuan Hu is also with the School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, as a visiting Ph.D. student. Thanks: Kuangji Zuo, Boyu Ma, Shihao Li, and Jianfei Yang are with the School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 (e-mail: [email protected]; [email protected]; [email protected]; [email protected]). Yuxuan Hu, Kuangji Zuo, Boyu Ma, Shihao Li, Zhaoyang Xia Abstract Reliable humanoid-robot interaction (HRI) in household environments is constrained by two fundamental requirements, namely robustness to unconstrained user positions and preservation of user privacy. Millimeter-wave (mmWave) sensing inherently supports privacy-preserving interaction, making it a promising modality for room-scale HRI. However, existing mmWave-based interaction-sensing systems exhibit poor spatial generalization at unseen distances or viewpoints. To address this challenge, we introduce WaveMan, a spatially adaptive room-scale perception system that restores reliable human interaction sensing across arbitrary user positions. WaveMan integrates viewpoint alignment and spectrogram enhancement for spatial consistency, with dual-channel attention for robust feature extraction. Experiments across five participants show that, under fixed-position evaluation, WaveMan achieves the same cross-position accuracy as the baseline with five times fewer training positions. In random free-position testing, accuracy increases from 33.00% to 94.33%, enabled by the proposed method. These results demonstrate the feasibility of reliable, privacy-preserving interaction for household humanoid robots across unconstrained user positions. I Introduction Humanoid robots have achieved substantial progress in perception, manipulation, and control in recent years [ 1 ] . However, in human-centered interaction scenarios, reliably understanding human intent while preserving user privacy remains a key bottleneck hindering their deployment in everyday environments. In real indoor settings, users may initiate interaction from arbitrary locations within a room, invalidating traditional assumptions of front-facing, short-range, or user-aligned interaction. Consequently, intent understanding must remain reliable under significant variations in distance, viewpoint, and spatial relationships [ 13 ] . Gestures constitute an intuitive human–robot interaction modality that conveys intent without speech or wearable devices, making them suitable for natural interaction in unconstrained spatial settings [ 6 ] . However, most existing gesture recognition systems are designed for short-range scenarios with fixed viewpoints, limiting their applicability in real environments [ 11 ] . In unconstrained human–robot interaction scenarios, the system cannot assume control over user pose or sensing viewpoint, and thus must rely on passively acquired observations from a fixed sensor. Accordingly, we consider an ambient sensing setting in which an environment-installed mmWave radar (e.g., a smart-home device) provides privacy-preserving observations to the robot for interaction. Under this setting, the relative distance, azimuth, and elevation between the user and the sensing node can vary substantially, imposing stricter requirements on the stability of room-scale gesture recognition across positions and viewpoints, as illustrated in Fig. 1 . Fig. 1: Spatially adaptive room-scale interaction scenario. WaveMan aligns observations from different user spatial positions into a unified perception space to mitigate spatial inconsistencies. Millimeter-wave (mmWave) radar has recently emerged as a promising sensing modality for room-scale gesture recognition [ 4 ] . Its non-intrusive sensing capability, robustness to lighting conditions and occlusion, and inherent privacy-preserving properties make it particularly suitable for long-term deployment in domestic environments to support humanoid interaction, offering clear advantages over vision-based [ 10 ] , audio-based [ 29 ] , and contact-based [ 19 ] systems. Prior studies have demonstrated that mmWave radar can capture fine-grained hand motion dynamics with high temporal resolution [ 24 ] . However, most existing mmWave-based gesture recognition methods remain limited to short-range or frontal configurations [ 22 ] , resulting in poor robustness under room-scale interaction conditions. Although recent efforts have explored long-range mmWave gesture recognition [ 17 ] , many of these approaches rely on raw ADC data, incurring high data throughput and bandwidth requirements that hinder practical deployment on resource-constrained in-home embedded sensing nodes or edge hubs. Point-cloud-based mmWave methods leverage chip-level preprocessing to extract more compact spatial representations, partially alleviating system-level data throughput and bandwidth constraints [ 21 , 25 ] . Existing approaches typically follow two paradigms: (i) end-to-end learning directly on raw or lightly processed point cloud sequences [ 21 ] , and (ii) transforming point cloud sequences into denser representations, such as time–frequency spectrograms, for deep learning [ 25 ] . Despite notable progress [ 27 ] , room-scale gesture recognition remains challenging. Substantial variations in user spatial position cause the same gesture to exhibit highly inconsistent observations across different distances and viewpoints, leading to pronounced feature distribution shifts and degraded generalization performance under unseen positions or viewpoints [ 15 , 14 ] . Moreover, gestures performed at long distances or off-axis directions typically produce weaker and sparser echoes with lower signal-to-noise ratios, further impairing the reliability of spectrogram-based representations [ 2 , 16 ] . Together, these factors induce geometry-, spectrum-, and direction-domain distribution shifts driven by spatial position changes, making existing methods ineffective under the random spatial configurations in domestic environments where the radar viewpoint is fixed. As a result, room-scale mmWave-based gesture recognition remains a largely unresolved challenge. To address these challenges, this paper proposes a spatially adaptive gesture recognition framework tailored for room-scale human–robot interaction. The main contributions of this work are summarized as follows: 1. A spatially adaptive room-scale perception framework for humanoid robot interaction. We develop a room-scale gesture interaction system for humanoid robots that integrates mmWave sensing with robot behavior execution, enabling stable interaction performance across varying user positions. 2. Geometry-, spectrum-, and direction-aware representation learning. We propose a spatially adaptive processing pipeline to address key spatial variations induced by user position changes in room-scale interaction scenarios. The pipeline consists of: (i) geometric alignment to compensate for position and viewpoint variations; (ii) unsupervised spectrogram enhancement to mitigate long-range sparsity and low signal-to-noise ratios; and (iii) an attention-based recognition network that fuses multi-channel information to improve robustness across positions. 3. Room-scale deployment and real-world humanoid robot evaluation. The proposed framework is deployed on a humanoid robot platform and evaluated under fixed-position, unseen-position, and random-position settings. Experimental results demonstrate reliable whole-room intent understanding, stable robot behavior execution, and significant performance improvements over existing methods. An overview of the entire framework is presented in Fig. 2 . The remainder of this paper is organized as follows: Section II reviews related work; Section III introduces the mmWave radar signal model and spectrogram derivation; Section IV presents the proposed spatially adaptive perception framework; Section V describes experimental setups and results; and Section VI concludes the paper. Fig. 2: Overview of the proposed spatially adaptive interaction framework. (a) Radar point-cloud data captured under diverse positional configurations are spatially aligned and transformed into spectrogram representations. (b) Sparse spectrograms are enhanced and fused with dense spectra to obtain robust gesture representations. (c) The recognized gestures are mapped to corresponding humanoid robot behaviors, enabling reliable human–robot interaction across unconstrained user positions. II Related Work II-A Gesture-based Human–Robot Interaction Gesture-based interaction

Read the complete text