SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge

Yumeng He, Ying Jiang, Jiayin Lu, Yin Yang, Chenfanfu Jiang · 2025

Read SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge by Yumeng He, Ying Jiang, Jiayin Lu, Yin Yang, Chenfanfu Jiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01629.txt.

Introduction

Yumeng He and co-authors rebuild articulated objects from a single photo for simulation. Objects with moving parts fill homes and workplaces. Robots need sim-ready models of them, with part trees and joints. Hand modeling those assets is slow expert work.

Their frame, SPARK, uses vision-language knowledge to infer parts. It outputs physically consistent, kinematic part models. Each part gets a place in a hierarchy with motion limits. One-photo assets could stock training worlds for robots far faster than hand modeling allows. Vision-language priors supply part names and joint guesses, which cuts the need for motion capture rigs in building each new asset.

Catalog record

CREATORYumeng He, Ying Jiang, Jiayin Lu, Yin Yang, Chenfanfu Jiang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01629

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge Yumeng He, Ying Jiang, Jiayin Lu, Yin Yang, Chenfanfu Jiang Abstract Articulated 3D objects are critical for embodied AI, robotics, and interactive scene understanding, yet creating simulation-ready assets remains labor-intensive and requires expert modeling of part hierarchies and motion structures. We introduce SPARK, a framework for reconstructing physically consistent, kinematic part-level articulated objects from a single RGB image. Given an input image, we first leverage VLMs to extract coarse URDF parameters and generate part-level reference images. We then integrate the part-image guidance and the inferred structure graph into a generative diffusion transformer to synthesize consistent part and complete shapes of articulated objects. To further refine the URDF parameters, we incorporate differentiable forward kinematics and differentiable rendering to optimize joint types, axes, and origins under VLM-generated open-state supervision. Extensive experiments show that SPARK produces high-quality, simulation-ready articulated assets across diverse categories, enabling downstream applications such as robotic manipulation and interaction modeling. Project page: https://heyumeng.com/SPARK/index.html . Figure 1 : SPARK is a novel framework that integrates VLM-guided part-level and global image guidance with diffusion transformers to produce high-quality articulated object reconstructions. † † * equal contribution. 1 UCLA, 2 USC, 3 University of Utah. [email protected] , jiayin_lu,yingjiang,[email protected] , [email protected] 1 Introduction Articulated objects are ubiquitous in everyday environments, and scalable creation of high-fidelity, interactable 3D assets is becoming increasingly important for embodied AI, robotics, and scene reconstruction. However, building articulated 3D models remains labor-intensive, often requiring manual labeling and expert modeling of part objects. With the advance of powerful generative models [ 28 , 21 ] , it is now possible to synthesize high-quality 3D assets directly from images or text prompts. However, the generated shapes are typically fused, making them difficult to reuse for downstream manipulation, animation, and simulation. Recent progress in part-level generation [ 22 , 3 , 47 ] enables the synthesis of semantically meaningful 3D parts, supporting fine-grained editing and control. Yet, despite strong geometric quality, these models often lack kinematic consistency because their part segmentation is driven purely by appearance, ignoring the underlying motion structure. Recently, articulated object generation methods [ 29 , 2 , 37 , 18 , 12 , 11 , 46 ] have sought to synthesize kinematic part-level objects together with their kinematic parameters, including joint and link properties. Many approaches [ 29 , 2 , 37 , 18 , 12 ] first generate or retrieve a 3D mesh and then segment it into movable parts under motion or multi-state image guidance, but these methods depend on template meshes or require multiple images for reliable segmentation. Other works [ 11 , 15 , 46 ] directly generate articulated objects using kinematic graphs as structural priors, while they require explicit kinematic parameters as input. For full URDF synthesis, both optimization-based [ 30 , 29 , 12 ] and feed-forward [ 20 , 18 , 6 , 55 ] methods have been proposed to estimate articulation parameters from multi-state images, videos, meshes, or reconstructed point clouds. To minimize user input, our goal is to reconstruct both kinematic part-level articulated objects and their corresponding URDF parameters from a single image. To address these challenges, we present SPARK, a framework that reconstructs simulation-ready articulated objects at the kinematic part level and estimates complete URDF parameters from a single RGB image. Given an input image, we first leverage vision–language models (VLMs) to extract coarse URDF parameters for joints and links. We then generate part-level images and construct a structure graph that encodes parent–child link relationships. Conditioning on both the structure graph and the synthesized part images, SPARK produces consistent part-level geometry and assembles them into a coherent articulated mesh through multi-level hierarchical attention. Finally, to further refine URDF parameters, we incorporate differentiable forward kinematics and differentiable rendering with a feature-injection strategy, optimizing joint attributes under the supervision of VLM-generated open-state images. To summarize, our contributions are: • We propose a novel framework that integrates a generative diffusion transformer with VLM priors to synthesize high-quality kinematic part-level articulated objects and accurately estimate the corresponding URDF parameters. • We introduce part-image guidance and multi-level attention to enable consistent multi-part synthesis, along with a joint optimization component that refines kinematic parameters under VLM-guided supervision. • Extensive experiments demonstrate that SPARK produces accurate and simulation-consistent articulated assets across diverse object categories. 2 Related Work 2.1 3D Object Generation 3D object generation aims to synthesize 3D shapes from texts, images, and point clouds, producing various representations such as triangle meshes [ 57 , 49 , 4 , 54 , 19 , 66 ] , analytic primitives [ 61 , 31 ] , point clouds [ 42 , 17 ] , signed distance fields (SDFs) [ 1 ] , neural radiance fields (NeRFs) [ 9 , 53 , 65 ] , and 3D Gaussian splats (3DGS) [ 40 , 48 , 41 , 5 , 62 ] . Among these, mesh generation is especially appealing for its high fidelity and compatibility with simulation and graphics pipelines [ 19 , 57 , 67 ] . To this end, TripoSG [ 19 ] and TRELLIS [ 57 ] employ flow-based models to generate high-fidelity 3D shapes, while Hunyuan3D [ 49 ] adopts a Diffusion Transformer (DiT) that first generates geometry and then recovers textures. However, the generated shapes are typically fused, requiring additional 3D segmentation. SAMPart3D [ 59 ] and PartDistill [ 51 ] adopt 2D-to-3D distillation strategies to segment 3D shapes from 2D priors, though the results are often incomplete and geometrically inconsistent in occluded regions. To further obtain complete 3D parts, HoloPart [ 58 ] employs a dual-attention diffusion model to complete partial segments, while PartField [ 26 ] leverages labeled 3D supervision to learn part features extending into the object interior. Instead of generating a complete shape followed by part extraction, recent approaches directly synthesize 3D parts from image latent representations [ 47 , 3 , 22 , 60 , 25 ] . PartComposer [ 25 ] and OmniPart [ 60 ] use spatial bounding boxes and 2D segmented part images, respectively, to guide part-level object generation with latent flow models and diffusion models. To further enhance structural guidance, AutoPartGen [ 3 ] exploits an autoregressive latent flow transformer to generate parts sequentially. In contrast, PartCrafter [ 22 ] and DualPacker [ 47 ] generate all parts simultaneously using a DiT with part-wise latents or a rectified-flow model with dual latent volumes. However, the resulting parts often lack kinematic awareness, leading to over- or under-segmentation. Our goal is to generate articulated objects at the kinematic-part level from a single image. 2.2 Articulated Object Shape Reconstruction Articulated object shape reconstruction focuses on recovering kinematic part-level geometry for each movable component. Prior works typically first obtain a complete 3D representation either by generating a whole 3D model [ 30 , 2 , 12 , 29 , 52 ] or by retrieving an existing 3D mesh [ 37 ] and then extract part-level articulated structures. Among these methods, Articulate AnyMesh [ 37 ] and DreamArt [ 30 ] explicitly segment the 3D representation using segmentation models [ 58 , 16 ] , making their results sensitive to segmentation quality. While FreeArt3D [ 2 ] , ArtGS [ 29 ]

Read the complete text