Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation

Jiahao Xu Xiaohan Yuan Xingchen Wu Chongyang Xu Kun Li Buzhen Huang · 2026

Read Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation by Jiahao Xu Xiaohan Yuan Xingchen Wu Chongyang Xu Kun Li Buzhen Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20336.txt.

Introduction

Xu and five colleagues propose a flow-matching frame for two humans sharing one object in joint handling. A generative model derives clear handling plans from the object's affordance and spatial layout, steering motion flow toward success. A rival interaction prior then fosters natural poses and true-to-life person-to-person play. A stability-led simulation refines shaky interaction states through sampling-based tuning and tunes vector field fits directly, beating human-object rivals on touch hit rate, overlap, and spread match.

Catalog record

CREATORJiahao Xu Xiaohan Yuan Xingchen Wu Chongyang Xu Kun Li Buzhen Huang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37473
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20336

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation Jiahao Xu Xiaohan Yuan Xingchen Wu Chongyang Xu Kun Li Buzhen Huang Abstract Co-manipulation requires multiple humans to synchronize their motions with a shared object while ensuring reasonable interactions, maintaining natural poses, and preserving stable states. However, most existing motion generation approaches are designed for single-character scenarios or fail to account for payload-induced dynamics. In this work, we propose a flow-matching framework that ensures the generated co-manipulation motions align with the intended goals while maintaining naturalness and effectiveness. Specifically, we first introduce a generative model that derives explicit manipulation strategies from the object’s affordance and spatial configuration, which guide the motion flow toward successful manipulation. To improve motion quality, we then design an adversarial interaction prior that promotes natural individual poses and realistic inter-person interactions during co-manipulation. In addition, we also incorporate a stability-driven simulation into the flow matching process, which refines unstable interaction states through sampling-based optimization and directly adjusts the vector field regression to promote more effective manipulation. The experimental results demonstrate that our method achieves higher contact accuracy, lower penetration, and better distributional fidelity compared to state-of-the-art human-object interaction baselines. The code is available at https://github.com/boycehbz/StaCOM . Figure 1 : Given an object mesh and its trajectory (green), our method generates coordinated motions that are consistent with the trajectory while remaining natural and physically plausible for co-manipulation. 1 Introduction Modeling coordinated human motion in interactive environments is a key problem in computer graphics, virtual reality, and robotics. The challenge intensifies in scenarios where multiple agents interact with shared objects ( e.g . , two-person lifting or collaborative manipulation), since each agent must adapt to both the object’s motion and the partner’s behavior. Such human-human co-manipulation involves tightly coupled triadic interactions, which require synchronized motion between humans and objects and physical feasibility during joint manipulation. However, most existing motion generation methods are tailored to single-person scenarios, where an individual moves in response to static environments or predefined object trajectories Li et al. 2023 ; Tevet et al. 2022 . These models lack the mechanisms for inter-agent communication and mutual adaptation, making them unsuitable for collaborative tasks in co-manipulation. On the other hand, recent multi-person motion generation methods Liang et al. 2024 ; Li et al. 2024c typically focus on social or dance-like interactions without involving object manipulation. Directly adapting these frameworks to co-manipulation cannot ensure reasonable physical dynamics and coherent human–object coordination. Consequently, developing motion generation models that can capture both inter-agent coordination and realistic human–object dynamics for multi-person co-manipulation remains an open research question. We observe that real-world co-manipulation motions need to satisfy three key aspects: 1) Intention : the manipulation strategy should be determined based on the object’s shape, affordance, and goal state to achieve the desired manipulation. 2) Naturalness : human motion should be natural and responsive to the partner’s actions. 3) Effectiveness : the transport process should be stable and comply with physical laws. Based on these observations, we propose a flow-matching framework that integrates object affordances, motion priors, and physics-based feedback into the motion generation process. Guided by these principles, our model generates co-manipulation motions that better satisfy intention, naturalness, and physical plausibility. Specifically, we adopt flow-matching Lipman et al. 2022 as our basic framework, which learns a continuous vector field to map noise into clean motion data under object trajectory guidance. We further employ BPS representation Prokudin et al. 2019 to enhance the model’s perception of dynamic object pose and shape. Building upon this, we introduce an affordance-informed manipulation strategy to guide motion generation during inference. Since humans can produce the same object motion through diverse movements, this strategy generates graspability fields conditioned on object affordances, and the resulting contact anchors serve as explicit gradient cues that attract hands toward high-probability regions on the object surface, yielding geometrically consistent yet diverse manipulation plans. Nonetheless, relying solely on the manipulation strategy often results in stiff or poorly synchronized poses. We therefore develop an adversarial interaction prior that scores dual-human motion naturalness by analyzing joint rotations, inter-agent timing, and role symmetry. In contrast to previous discriminators Kanazawa et al. 2018 ; Goel et al. 2023 , this prior focuses on collaborative cues and penalizes misaligned reactions, so the flow receives gradients that preserve coordinated behaviors. In addition, we incorporate a stability-driven simulation to inject physics-based feedback, where a sampling-based formulation refines unstable poses and steers the generator toward motions that maintain grasp stability and limit payload drift. The main contributions of this work are summarized as follows: • We propose a flow-matching framework to consider intention, naturalness, and effectiveness principles to generate physically plausible and socially coordinated human co-manipulations. • We introduce an affordance-informed manipulation strategy and an interaction prior that ensure natural interactions while producing plausible manipulations. • We consider co-manipulation motion with a focus on stability and propose a sampling-based simulation to jointly refine the interaction and motion. 2 Related Work Figure 2 : Overview. Given an input object trajectory, our method generates co-manipulation motions conditioned on object 6D poses and their BPS features (a). To ensure that the motions are consistent with the object trajectory, an affordance-informed manipulation strategy (b) is introduced to produce explicit contact signals as flow guidance. Building on this design, we further propose an adversarial interaction prior (c) and a stability-driven simulation (d) to enhance motion quality. Note that the contact strategy (b), flow matching (a), and interaction prior (c) are trained separately in advance, while all components are executed jointly at inference time. Human-Object Interaction Generation. Human-object interaction (HOI) generation has evolved from contact-aware diffusion to affordance-driven reasoning, yet most methods still emphasize single-actor scenes Xu et al. 2025b ; Wu et al. 2025c ; Pi et al. 2023 ; Li et al. 2024b ; Liu et al. 2024 ; Wu et al. 2025a . InterDiff Xu et al. 2023 and CG-HOI Diller and Dai 2024 inject contact objectives into diffusion pipelines, whereas OMOMO Li et al. 2023 , NIFTY Kulkarni et al. 2024 , and NAP Lei et al. 2023 reconstruct interactions from object trajectories or articulated priors to provide semantic control. With the availability of advanced human–object–human datasets Liu et al. 2025b ; Xu et al. 2025a , recent works have begun to explore collaborative manipulation. OnlineHOI Ji et al. 2025b introduces a Mamba-based network for generating human motions conditioned on object geometry. SyncDiff He et al. 2025 promotes phase-aligned multi-body motion, while COLLAGE Daiya et al. 2025 samples cooperative contacts for dual agents. However, these methods rely solely on stochastic denoising without incorporating explicit physical feedback. CooHOI Gao et al. 2024 achieves physically plausible cooperative human–object interactions through reinforcement learning, but its policy fails to generalize across different objects and tasks. In contrast to prior works, our approach advances human co-manipulation motion generation by integrating deterministic flow matching with affordance-guided gradients and adversarial coordination priors. A generalizable sampling-based simulation module is further introduced to improve manipulation stability. Multi-person Interaction Generation. Motion generation for multi-person interaction focuses on modeling dynamic interactions between individuals Zhu et al. 2025 ; Ji et al. 2025a ; Xu et al. 2025c ; Ota et al. 2025 ; Chang et al. 2025 . However, existing methods have long inference times that are

Read the complete text