ARFlow: Human Action-Reaction Flow Matching with Physical Guidance

Wentao Jiang, Jingya Wang, Kaiyang Ji, Baoxiong Jia, Siyuan Huang, Ye ShiShanghaiTech UniversityBeijing Institute for General Artificial Intelligence · 2025

Read ARFlow: Human Action-Reaction Flow Matching with Physical Guidance by Wentao Jiang, Jingya Wang, Kaiyang Ji, Baoxiong Jia, Siyuan Huang, Ye ShiShanghaiTech UniversityBeijing Institute for General Artificial Intelligence (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16973.txt.

Introduction

Wentao Jiang and colleagues present a method for generating how a person reacts to another person action, for uses such as virtual reality and social robots. Earlier diffusion-based systems built reactions from noise through tangled conditioning steps, and their motions often broke physics, with bodies passing through each other. Writing in twenty twenty-five, the team instead learns a direct flow from an action to its reaction. A physical guidance step steers sampling away from body overlap, and a corrected sampling scheme plus added randomness improve both realism and variety.

Tests on three public human interaction data sets showed better motion quality and more varied reactions than prior methods. Collisions dropped sharply under new measures the authors introduce for overlap volume and frequency. Readers get a clear example of how adding simple physical guardrails can fix a common failure of generative motion models.

Catalog record

CREATORWentao Jiang, Jingya Wang, Kaiyang Ji, Baoxiong Jia, Siyuan Huang, Ye ShiShanghaiTech UniversityBeijing Institute for General Artificial Intelligence
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16973

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ARFlow: Human Action-Reaction Flow Matching with Physical Guidance Wentao Jiang, Jingya Wang, Kaiyang Ji, Baoxiong Jia, Siyuan Huang, Ye ShiShanghaiTech UniversityBeijing Institute for General Artificial Intelligence Abstract Human action-reaction synthesis, a fundamental challenge in modeling causal human interactions, plays a critical role in applications ranging from virtual reality to social robotics. While diffusion-based models have demonstrated promising performance, they exhibit two key limitations for interaction synthesis: reliance on complex noise-to-reaction generators with intricate conditional mechanisms, and frequent physical violations in generated motions. To address these issues, we propose Action-Reaction Flow Matching (ARFlow), a novel framework that establishes direct action-to-reaction mappings, eliminating the need for complex conditional mechanisms. Our approach introduces a physical guidance mechanism specifically designed for Flow Matching (FM) that effectively prevents body penetration artifacts during sampling. Moreover, we discover the bias of traditional flow matching sampling algorithm and employ a reprojection method to revise the sampling direction of FM. To further enhance the reaction diversity, we incorporate randomness into the sampling process. Extensive experiments on NTU120, Chi3D and InterHuman datasets demonstrate that ARFlow not only outperforms existing methods in terms of Fréchet Inception Distance and motion diversity but also significantly reduces body collisions, as measured by our new Intersection Volume and Intersection Frequency metrics. Our project is available at https://arflow2025.github.io/ . 1 Introduction Human action-reaction synthesis [ 59 , 72 , 10 ] has emerged as a pivotal research direction in computer vision [ 70 , 39 , 58 , 28 , 60 , 67 ] . This task aims to generate physically plausible human reactions responding to observed actions, with critical applications in virtual reality, human-robot interaction, and character animation. Unlike single-human motion generation [ 19 , 62 , 8 ] , reactors must infer responses without observing future actor motions, creating unique modeling challenges. While recent diffusion methods [ 62 , 72 ] show promise in motion generation, they face two key limitations in the modeling of action-reaction interactions. First, existing approaches [ 62 , 72 ] indirectly model responses using noise-to-reaction generators with intricate conditional mechanisms like treating action information as a condition to guide the generation process. This not only complicates the training process but also fails to directly capture the inherent causal relationship between actions and reactions. Second, frequent physical violations like body penetration between characters occur due to neglected physical constraints. While such issues are absent in single-human scenarios, they become critical in human interaction applications [ 26 ] . This poses a significant barrier to real-world applications such as virtual reality and human-robot interaction, where even minor physical inaccuracies are intolerable [ 53 , 26 ] . To address these challenges, we propose Action-Reaction Flow Matching (ARFlow), a novel framework that fundamentally resolves these limitations. Unlike diffusion models constrained by noise-data mappings, flow matching [ 40 , 44 , 1 , 48 ] naturally models paired distributions through linear interpolation between endpoints (See Fig. 1 ), making it inherently suitable for action-reaction synthesis. ARFlow establishes direct pathways between action and reaction distributions, enabling simpler training and faster inference compared to diffusion-based methods. Instead of traditional vector field prediction ( \mathbf{v} -prediction) in standard flow matching [ 40 , 27 ] , ARFlow directly outputs clean body poses ( \mathbf{x}_{1} -prediction) in human motion generation. To eliminate unrealistic body collisions between characters, we further develop a specialized sampling algorithm that performs physical constraint guidance at \mathbf{x}_{1} to avoid inaccurate gradients, and projects it back onto a revised Flow Matching path through interpolation. This innovation maintains physical plausibility through gradient guidance without compromising motion quality. Additionally, we introduce two new physics-aware evaluation metrics: Intersection Volume (IV) and Intersection Frequency (IF). Our main contributions are threefold: • We propose ARFlow, the first Flow Matching architecture that creates direct pathways between human action and reaction distributions, eliminating the need for complex conditional mechanisms prevalent in diffusion approaches. • We integrate physical constraints and employ a reprojection method to revise the sampling direction of FM, surpassing vanilla guidance methods. This successfully prevents body collisions between characters while preserving natural movement. To further enhance the reaction diversity, we introduce randomness into the sampling process. • We introduce two new evaluation metrics (IV, IF) for interaction quality assessment. Experiments demonstrate state-of-the-art performance in Fréchet Inception Distance (FID), MultiModality (Multimod.), IV and IF on NTU120 and Chi3D datasets. Figure 1: Our proposed Human Action-Reaction Flow ( ARFlow ). We directly establish a mapping between the action and reaction distribution and our sampling process is further guided by our physical constraint guidance. The change of colors represents the variation of the h -frame reaction with sampling timestep t_{n} . 2 Related Work 2.1 Human Motion Generation Human motion synthesis aims to generate diverse and realistic human-like motion conditioned on different guidances, including text [ 79 , 21 , 33 , 14 ] , music [ 83 , 37 ] , speech [ 22 , 4 ] , sparse signals [ 15 , 61 ] , action labels [ 19 , 50 , 62 , 6 , 8 ] , or unconditioned [ 51 , 62 , 74 ] . Recently, many diffusion-based motion generation models have been proposed [ 78 , 62 , 14 , 8 , 69 , 63 ] and demonstrate better quality compared to alternative models such as VAE [ 34 , 19 , 50 , 6 ] , flow-based models [ 54 , 3 ] or GANs [ 74 , 70 ] . Alternatively, motion can be regarded as a new form of language and embedded into the language model framework [ 77 , 29 ] . Meanwhile, the exploration of guiding the sampling process of diffusion models [ 12 , 75 ] has been a key area in motion diffusion models, PhysDiff [ 31 ] proposes a physics-guided motion diffusion model, which incorporates physical constraints in a physics simulator into the diffusion process. GMD [ 31 ] presents methods to enable spatial guidance without retraining the model for a new task. DNO [ 31 ] proposes a motion editing and control approach by optimizing the diffusion latent noise of an existing pre-trained model. 2.2 Human Action-Reaction Synthesis Different from human-human interaction [ 70 , 39 , 58 , 28 , 60 , 67 ] , human action-reaction synthesis is causal and asymmetric [ 72 , 42 , 71 ] . To address this task, researchers have leveraged large language models [ 57 , 59 , 30 ] , VAE-based methods [ 10 , 45 , 46 ] . However, these methods cannot capture fine-grained representations and ensure diversity, and diffusion-based methods [ 38 , 60 ] are limited to the “offline” and “constrained” setting of human reaction generation, failing to generate instant and intention agnostic reactions. More recently, ReGenNet [ 72 ] introduce a diffusion-based transformer decoder framework and treat action sequence as conditional signal for online reaction generation. However, it often produce physically-implausible inter-penetrations between the actor and reactor since they disregard physical constraints in the generative process. Our method addresses this problem by ARFlow sampling with physical constraint guidance. 2.3 Flow Matching Flow Matching [ 40 , 44 , 1 , 48 , 47 , 17 ] has emerged as an efficient alternative

Read the complete text