PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE

He Zhang, Tao Yu, Qiu Shen, Xun Cao · 2026

Read PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by He Zhang, Tao Yu, Qiu Shen, Xun Cao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26741.txt.

Introduction

He Zhang, Tao Yu, Qiu Shen, and Xun Cao present PressMimic, a framework that teaches humanoid robots to imitate people using pressure. Vision-only motion capture misses contact physics. Robots trained on it slide their feet, sink through floors, and lose balance. Pressure under the feet records exactly when and how contact happens. The authors use it in both perception and control.

A perception model fuses camera images with pressure to estimate three-dimensional pose and global motion. Pressure supplies contact constraints that resolve visual ambiguity. On the control side, a pressure-supervised policy learns from those contact signals during reinforcement learning. The robot reproduces physically consistent contact patterns. The team also built MotionPRO, a large data set synchronising video, pressure, and motion capture. Experiments showed pressure improves both estimation and imitation quality.

Catalog record

CREATORHe Zhang, Tao Yu, Qiu Shen, Xun Cao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26741

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE He Zhang, Tao Yu, Qiu Shen, Xun Cao Abstract Humanoid motion imitation requires not only accurate perception of human kinematics but also faithful reproduction of physical interactions with the environment. However, existing pipelines rely primarily on vision-based motion capture and kinematic imitation, largely ignoring contact dynamics, leading to artifacts such as foot sliding, floor penetration, and unstable behaviors. In this work, we revisit humanoid motion imitation from the perspective of physical grounding and leverage pressure as a unified modality across perception and control. We present PressMimic, a framework that integrates pressure into the full pipeline from motion capture to humanoid control. In the perception stage, we introduce FRAPPE + ⁣ + \text{FRAPPE}^{++} , a multimodal model that fuses RGB and pressure to jointly estimate 3D pose and global motion, where pressure provides explicit contact and support constraints to resolve ambiguity in vision-based estimation. In the control stage, we propose a pressure-supervised policy (PSP) that incorporates pressure-derived signals into reinforcement learning, enabling physically consistent contact patterns during execution. We further construct MotionPRO, a large-scale dataset with synchronized RGB, pressure, and motion capture data. Experiments show that pressure improves motion estimation accuracy, trajectory consistency, and execution stability. These results demonstrate that pressure serves as an effective physical grounding signal, bridging perception and control for physically consistent humanoid motion imitation. Project Page: https://yeelou.github.io/PressMimic/ I Introduction Enabling humanoid robots to faithfully replicate human motion has long been a central challenge at the intersection of computer vision and robotics. Achieving this goal demands not only accurate perception of human body kinematics, but also robust translation of such perception into physically plausible robot control. Despite remarkable progress in both human motion capture [ 26 , 61 , 32 , 35 , 17 , 33 , 19 , 24 , 70 , 14 ] and humanoid motion control [ 20 , 67 , 7 ] , existing approaches remain agnostic to the physical interactions between the body and its environment, leading to failure modes including temporal jitter, floor penetration, and foot-sliding artifacts. This perceptual deficiency creates a critical bottleneck: robots driven by kinematically-estimated references frequently fail to reproduce the physical grounding of human motion, resulting in unstable locomotion, degraded performance in scenarios with frequent contact changes, and, in extreme cases, hardware damage. Pressure naturally encodes the foot-ground interactions and contact dynamics that kinematic signals fail to capture. Previous work [ 52 ] demonstrated that pressure substantially improves MoCap quality by providing explicit grounding constraints on 3D pose and trajectory estimation. Beyond perception, pressure also offers a direct supervisory signal for robot motion control, constraining contact timing, ground reaction forces, and weight distribution that existing imitation pipelines overlook. In this work, we present PressMimic , a unified framework that leverages pressure as a core modality across the full pipeline of humanoid motion imitation from human motion capture to robot motion control. To the best of our knowledge, PressMimic is the first system to systematically explore and exploit pressure signals in both the perception and control stages of humanoid motion imitation. The proposed framework operates in two stages. In the first stage, we introduce a multimodal human motion estimation network \text{FRAPPE}^{++} that fuses RGB video with pressure maps. Both modalities are non-invasive and require no wearable instrumentation, yet capture complementary aspects of human motion: RGB encodes visual appearance and geometric structure, while pressure provides explicit contact states and ground reaction dynamics imperceptible to cameras alone. To exploit this complementarity, we extract compact pressure representations and model spatiotemporal dependencies across modalities, fusing them into a unified motion representation. We further impose orthographic similarity constraints along the camera axis and whole-body contact constraints along the vertical axis to combine precise lower-body contact and global translation cues from pressure with accurate full-body pose from RGB. In the second stage, we introduce a Pressure-Supervised motion control Policy ( PSP ), where plantar pressure maps are modeled as distributional offsets characterizing foot contact patterns and incorporated as an auxiliary reward term during reinforcement learning, explicitly encouraging the robot to reproduce the ground reaction dynamics of the human demonstrator. To support both stages, we introduce MotionPRO , a large-scale multimodal dataset capturing pressure, RGB video, and optical motion capture signals from 70 volunteers performing 400 motion types, encompassing 12.4M pose frames in total. Extensive experiments on MotionPRO demonstrate that our multimodal fusion approach outperforms state-of-the-art RGB-based pose and trajectory estimation methods, maintaining physically plausible estimates even under extreme occlusions and severe vertical trajectory drift. For humanoid motion imitation, pressure supervision consistently improves task success rates and locomotion stability while preserving strong motion similarity to the reference demonstrations. The main contributions of this work are summarized as follows: • We present PressMimic , the first unified framework to systematically exploit pressure across both the perception and control stages of humanoid motion imitation. • We propose a multimodal motion estimation network \text{FRAPPE}^{++} fusing RGB and pressure, achieving robust pose and trajectory estimation under occlusion and trajectory drift. • We introduce a pressure-supervised control policy called PSP that incorporates pressure-derived distributional offsets as an auxiliary reward, improving task success rates and locomotion stability in humanoid motion imitation. • We construct MotionPRO , a large-scale human motion dataset with pressure, RGB, and optical sensors. Extensive experiments demonstrate that our methods can achieve significant performance improvements in both motion estimation and motion imitation tasks. II Related Work II-A Humanoid Robot Motion Imitation Humanoid robot motion imitation aims to enable robots to reproduce human motion in a physically plausible and stable manner. Early approaches [ 31 , 1 , 15 , 4 , 48 , 50 , 13 , 12 ] relied on motion retargeting to directly map human joint configurations to robot joint angles, followed by trajectory optimization or model predictive control to generate feasible robot motions. While effective for simple periodic motions such as walking, these methods struggle to generalize to complex or contact-rich behaviors due to their reliance on accurate dynamic models and hand-crafted controllers. Reinforcement learning (RL) has emerged as the dominant paradigm for humanoid motion imitation, enabling controllers to be learned directly from interaction with physics simulators. DeepMimic [ 51 ] pioneered the use of reference-state initialization and motion imitation rewards to train physics-based controllers capable of reproducing a wide range of human motions. Subsequent works extended this framework to handle more diverse motion repertoires [ 42 , 43 ] , multi-skill transitions [ 60 , 41 ] , and real-world deployment on physical robots [ 68 , 77 , 44 , 23 , 66 , 22 , 34 , 74 ] . PHC [ 42 ] introduced a perpetual humanoid controller that tracks arbitrary motion sequences through curriculum-based training. More recently, BeyondMimic [ 36 ] and SONIC [ 44 ] have advanced whole-body motion imitation on full-sized humanoid platforms, demonstrating stable policy deployment on real hardware. Despite these advances, existing methods construct their imitation rewards purely from kinematic references, including joint positions, orientations, and end-effector states, without accounting for the dynamic foot-ground interactions that underpin stable locomotion. This limitation frequently manifests as foot-sliding, floor penetration, and unstable gait patterns, particularly in contact-rich scenarios. In contrast, PressMimic incorporates plantar pressure

Read the complete text