Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki · 2026

Read Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning by Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11346.txt.

Introduction

Yuto Shibata and coauthors teach humanoid agents to assist a human partner through force-exchanging contact. Caregiving robots must give this help, but motion tracking alone cannot. They cast imitation of assistive human-human motion as multi-agent reinforcement learning. Supporter and recipient train together inside a physics simulator. Priors from single-human tracking controllers ease exploration. Dynamic reference retargeting adapts its motion to the partner's live pose. A contact-promoting reward pushes for meaningful support. It is the first to track assistive interactions on established benchmarks.

Catalog record

CREATORYuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57089
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11346

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki Abstract Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human–human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant’s reference motion to the recipient’s real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control. 1 Introduction Recent advances in humanoid control and character animation have been driven by motion imitation from human motion capture data 2 ; 30 ; 35 ; 10 ; 23 . Replacing hand-engineered task rewards 36 with motion imitation provides a powerful and general reward formulation that minimizes reward design effort while naturally defining task objectives for both robots and virtual characters. Retargeting human motion to humanoid robots and training tracking-based controllers have enabled agile, robust, and disturbance-resistant behaviors 7 ; 5 ; 2 ; 34 ; 29 . Despite these advances, existing tracking-based controllers primarily focus on single-person motion sequences 10 . In contrast, interactive behaviors involving close contact and force exchange, which are central to assistive and caregiving scenarios, remain a major open challenge. Learning such behaviors requires not only reproducing reference motions but also continuously adapting to a partner’s changing dynamics. Prior work typically addresses multi-agent interaction with a kinematic replay strategy: a pre-trained single-agent controller generates the recipient’s motion in isolation, which is then played back while the assistive agent learns to react 6 ; 9 . However, this approach cannot be applied to scenarios in which multiple agents move simultaneously while influencing each other. Indeed, in tightly coupled assistive motions, the recipient’s behavior cannot be computed independently, and decoupling the learning of partner control policies breaks physical consistency. We address this challenge with AssistMimic, a multi-agent reinforcement learning (MARL) framework for learning physics-aware, tracking-based controllers for close human–human interactions. Unlike prior approaches that rely on kinematic replay, AssistMimic jointly trains policies for both partners within a MARL formulation 31 ; 4 . This joint optimization removes the need to “freeze” one agent’s motion, allowing the assistant and recipient to co-adapt during training and enabling rich bidirectional coordination, particularly when the recipient becomes physically unstable. Yet, reinforcement learning for reactive control in physically entangled interactions is highly challenging. As shown in Figure 2 , unlike the contact-less social interactions or isolated motions examined in prior work such as high-fives (top), contact-rich assistive behaviors (bottom) require precise spatial alignment and tightly timed force application; even minor errors in contact location or force magnitude can destabilize the recipient. The difficulty is compounded by severe occlusion in close-range human–human motion capture, yielding noisy and inaccurate reference trajectories. These factors make the MARL training unstable. AssistMimic facilitates exploration in MARL through three core components. It initializes partner policies from single-person tracking controllers to leverage strong locomotion priors, uses dynamic reference retargeting to maintain stable close-contact interactions, and incorporates contact-promoting rewards to enable learning from noisy or incomplete demonstrations. Together, these innovations enable stable and physically realistic imitation of interactive assistive motions in simulation, leading to successful learning, as shown by the orange curve in Figure 2 . Figure 2 : Learning contact-rich assistive behaviors is substantially more difficult in the close-contact interactions that we target (bottom) than in contact-less social interactions (top) or isolated motions ( gray SR curve). AssistMimic addresses these challenges, achieving successful imitation for the first time, as shown in the improved orange SR curve. We evaluate AssistMimic on the Inter-X 26 and HHI-Assist 19 datasets, both of which feature tightly coupled, support-oriented human–human interactions. Across these benchmarks, AssistMimic is the only method to track closely interacting, force exchanging human motions, achieving substantially higher task success rates (Inter-X: 83%, HHI-Assist: 66%) and demonstrating strong robustness to unseen disturbances such as stumbles and load shifts. Empirically, AssistMimic accurately reproduces reference motions across diverse assistive scenarios—including recipients lying on the floor, seated in chairs, or resting in beds—where the assisting agent learns to provide appropriate physical support and lifting assistance. To objectively assess assistance quality, we reduce the recipient’s PD gains and maximum torque limits to constrain self-stabilization, thereby isolating the contribution of the learned assistive controller. In summary, our key contributions are as follows: • Formulation: We cast physics-based human–human motion imitation as a multi-agent reinforcement learning (MARL) framework for learning physics-enabled tracking controllers, enabling motion imitation for caregiving and assistive tasks that require reactive force exchange. • Methodology: We further introduce motion prior initialization, dynamic reference retargeting, and contact-promoting reward schemes that make multi-agent RL tractable and effective in high-contact settings. • Evaluation: We conduct extensive experiments and ablations that quantify each component’s contribution to learning efficiency, imitation fidelity, and assistive stability. 2 Related Work Physics-Based Human Motion Synthesis. Early work on physics-based character control, such as DeepMimic 15 , demonstrated that reinforcement learning (RL) can reproduce diverse motion sequences within physically realistic simulators 13 ; 16 . Follow-up frameworks, including AMP 17 and PHC 10 , expanded this paradigm to generate more stable and diverse behaviors through adversarial rewards and goal-conditioned training 12 ; 33 . More recent approaches, such as PhysDiff 32 , integrate diffusion-based generative models with physics simulation to improve realism and stability. However, these efforts primarily focus on single-agent motion and remain limited in modeling the reactive forces and bidirectional dependencies inherent in multi-agent interactions targeted in our work. Human–Human and Human–Robot Interaction Synthesis. The study of human–human interaction synthesis has evolved rapidly, with recent diffusion-based 8 ; 28 ; 22 and transformer-based 27 ; 3 architectures enabling context-aware multi-agent motion generation. While these models effectively capture social cues and motion continuity, they are typically trained in kinematic space and thus lack physical grounding. The recent Human-X framework 6 takes a step toward bridging this gap by integrating an autoregressive diffusion planner with a physics-tracking policy to achieve real-time, physically plausible interactions across human–avatar, human–humanoid, and human–robot domains. Yet, its motion control remains largely reactive and open-loop, relying on pretrained diffusion priors rather than continuous bidirectional adaptation, in which both agents generate motions conditioned on each other’s states. Multi-Agent Reinforcement Learning for Physical Interaction. Multi-agent reinforcement learning (MARL) has shown promise for modeling interactive behaviors in physically coupled settings 31 ; 4 . Within humanoid control, however, most MARL methods focus on competitive or

Read the complete text