ArticFlow: Generative Simulation of Articulated Mechanisms
Jiong Lin, Jinchen Ruan, Hod Lipson · 2025
Read ArticFlow: Generative Simulation of Articulated Mechanisms by Jiong Lin, Jinchen Ruan, Hod Lipson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17883.txt.
Introduction
Jiong Lin, Jinchen Ruan, and Hod Lipson introduce ArticFlow, a generative method for mechanisms with moving joints, such as robot bodies and hinged objects. Generating static shapes is now strong, but motion dependent deformation and scarce data make articulated shapes much harder.
ArticFlow uses two linked flow stages: one moves noise to a compact shape code, and the other moves points under an explicit action, guided by that code. One model can cover many mechanism types and generalize across actions. On a standard robot collection it beats object specific simulators in motion accuracy and shape quality, and it can invent new forms by mixing codes.
Catalog record
| CREATOR | Jiong Lin, Jinchen Ruan, Hod Lipson |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-17883
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ArticFlow: Generative Simulation of Articulated Mechanisms Jiong Lin, Jinchen Ruan, Hod Lipson Abstract Recent advances in generative models have produced strong results for static 3D shapes, whereas articulated 3D generation remains challenging due to action-dependent deformations and limited datasets. We introduce ArticFlow, a two-stage flow matching framework that learns a controllable velocity field from noise to target point sets under explicit action control. ArticFlow couples (i) a latent flow that transports noise to a shape-prior code and (ii) a point flow that transports points conditioned on the action and the shape prior, enabling a single model to represent diverse articulated categories and generalize across actions. On MuJoCo Menagerie, ArticFlow functions both as a generative model and as a neural simulator: it predicts action-conditioned kinematics from a compact prior and synthesizes novel morphologies via latent interpolation. Compared with object-specific simulators and an action-conditioned variant of static point-cloud generators, ArticFlow achieves higher kinematic accuracy and better shape quality. Results show that action-conditioned flow matching is a practical route to controllable and high-quality articulated mechanism generation. Figure 1 : ArticFlow models a category of diverse articulated objects with a single point-flow field, conditioned on shape prior and joint actions. In this figure, columns interpolate the shape-prior condition (novel instances); rows sweep the action control (joint angles). Examples include eyeglasses from PartNet-Mobility 53 and robot arms from MuJoCo Menagerie 49 . The same field produces coherent deformations for one category of articulated shapes, enabling interpolation and controllable 3D generation. 1 Introduction Generative models have become a leading approach for content generation across images, audio, and video. The model landscape has progressed from early variational autoencoders (VAEs) 19 and generative adversarial networks (GANs) 13 to flow based methods such as continuous normalizing flows (CNFs) 44 ; 39 , diffusion models 14 ; 47 , and, more recently, flow matching methods 23 . Building on this progress, recent work has increasingly turned to 3D content 10 ; 9 , where geometric structures and physical constraints introduce additional challenges. Broadly, two competing paradigms have emerged for 3D generation. The first leverages powerful 2D priors to synthesize multi-view consistent images and then lifts them to 3D; for example, Zero-1-to-3 27 ; 46 style 3D synthesis pipelines. This family benefits from 2D foundation models 45 pre-trained on high-quality image datasets, but often struggles with true 3D consistency and geometric fidelity. They typically rely on a separate reconstruction stage 31 to convert images into watertight shapes with accurate collisions and physical properties. The second paradigm performs explicit 3D generation, directly producing point clouds 54 ; 56 ; 52 , meshes, or voxels. It depends on high-quality, curated 3D datasets 3 ; 32 and typically demands model designs 41 ; 42 that align with representation-specific properties (e.g., permutation invariance for point clouds, connectivity constraints for meshes, and spatial sparsity for voxels). However, many real-world objects are articulated, coupling geometry with kinematics; this setting stresses both paradigms and motivates dedicated generative models for articulated shapes. Within this articulated setting, interest is rising, but generative methods remain in their early stages. Most prior work focuses on analysis, such as part segmentation, articulation inference, and motion estimation 21 ; 22 . The challenge is to ensure 3D consistency, along with kinematic correctness and plausible deformation. Existing approaches either condition on a given image 24 ; 5 and then reconstruct its articulation, or they follow a multi-stage pipeline that first generates a kinematic graph (parts, joints, parameters) and subsequently synthesizes geometry for each part 26 . Against this backdrop, we propose a self-contained, 3D-explicit solution for articulated mechanism generation. We represent each object as a deformable point cloud and learn an action conditioned flow matching velocity field that captures kinematic motion from sets of deformation samples. We further condition the field on a shape prior, enabling a single model to capture the kinematics of an entire category of articulated shapes. By interpolating in the action and shape-prior space, the model generates novel instances and motions while maintaining 3D consistency and kinematic validity. As shown in Figure 1 , we evaluate across multiple categories from the PartNet-Mobility 53 dataset and on simulated robots from MuJoCo Menagerie 49 . Our model exhibits disentangled conditioning: with the shape prior held fixed, generated samples preserve appearance while producing kinematically correct deformations; with the action held fixed, generated samples maintain a similar pose while varying appearance to yield novel instances. Beyond visual novelty, these results point to a broader opportunity: the potential to address the long-standing scarcity of diverse robot morphologies. High quality articulated objects and simulation ready robot models are expensive. For example, PartNet-Mobility includes 26 folding chairs, and only 17 robot arms are available in MuJoCo Menagerie . Meanwhile, robot behavior datasets have scaled from RT-1 to RT-X 2 ; 57 ; 38 , reaching 10^{5} tasks with millions of trajectories, yet the number of distinct robot embodiments is 22, creating a morphology bottleneck. Viewing robots as autonomous articulated bodies, we explore whether generative models can produce novel embodiments with valid kinematics and meaningful morphology. Our contributions: (1) We introduce an action conditioned flow matching framework that learns a velocity field in point space to generate deformations under given joint commands. (2) We extend the framework with joint conditioning on action and shape prior, enabling a single model to cover a category of articulated shapes and generate unseen instances with kinematically valid deformations. (3) We evaluate on everyday objects and robots, demonstrating controllable generation, interpolation across instances, and accurate kinematic behavior under novel action sequences. 2 Related Work Method Model Shape Latent Articulated Color Generation Interpolation Deformation Support VSM 4 DeepSDF ✗ ✗ ✓ ✗ FFKSM 15 NeRF ✗ ✗ ✓ ✗ l-GAN 1 GAN ✓ ✓ ✗ ✗ PointFlow 54 CNF ✓ ✓ ✗ ✗ PVD 56 Diffusion ✓ ✗ ✗ ✗ Point-E 35 Diffusion ✓ ✗ ✗ ✓ PSF 52 Flow Matching ✓ ✗ ✗ ✗ ArticFlow Flow Matching ✓ ✓ ✓ ✓ Table 1 : Compare with previous methods. Rows 1–2 are neural simulation methods. Rows 3–7 are point cloud generation models. 2.1 Point cloud generation Early methods adapted GAN and VAE to unordered point sets, for example, l-GAN 1 and hierarchical set-structured Set-VAE 18 . Flow based approaches model continuous deformations in data space: PointFlow 54 learns a continuous normalizing flow over point clouds (trained on ShapeNet) by coupling a latent prior with a point-space flow. Diffusion models followed, including point-voxel or pure point formulations such as PVD 56 , PSF 52 , and DiT-3D 33 , which leverage score driven denoising with set-aware backbones. Latent diffusion has also been explored: LION 50 trains an autoencoder for point sets and performs diffusion in the learned latent. Beyond unconditional generation, Point-E 35 conditions on 2D image priors or text to produce colored point clouds, while the autoregressive tokenization of point sets (e.g., PointGPT 6 ) treats generation as next-token prediction with permutation-robust tokenizers. These methods improve fidelity and diversity while addressing set invariance, geometric realism, and efficient editing and inpainting. As generative models have grown in capability, architectural design has evolved from two-stage conditional generation, as in l-GAN and PointFlow, to direct one-stage formulations, such as the DDPM-based PVD and the rectified flow-matching framework of