SplArt : Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting

Shengjie Lin, Jiading Fang, Muhammad Zubair Irshad, Vitor Campagnolo Guizilini, Rares Andrei Ambrus, Greg Shakhnarovich, Matthew R. Walter · 2025

Read SplArt : Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting by Shengjie Lin, Jiading Fang, Muhammad Zubair Irshad, Vitor Campagnolo Guizilini, Rares Andrei Ambrus, Greg Shakhnarovich, Matthew R. Walter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03594.txt.

Introduction

Shengjie Lin and co-authors present SplArt, a system that rebuilds movable objects in three dimensions and works out how their parts move. Their paper appeared in 2025. Objects such as laptops and drawers matter for robots and augmented reality. Past methods needed costly labels or fell into wrong answers. SplArt needs only two sets of photos of the object, taken in two poses.

The system builds the object from small soft blobs, a fast rendering method, and gives each blob a mobility score. A staged process first rebuilds shape, then splits parts, then estimates the joints. It learns from geometry alone, with no labels and no category rules. Tests on standard and new benchmarks, plus real scenes, showed strong accuracy. The rebuilt objects render in real time from fresh viewpoints and poses.

Catalog record

CREATORShengjie Lin, Jiading Fang, Muhammad Zubair Irshad, Vitor Campagnolo Guizilini, Rares Andrei Ambrus, Greg Shakhnarovich, Matthew R. Walter
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03594

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SplArt : Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting Shengjie Lin, Jiading Fang, Muhammad Zubair Irshad, Vitor Campagnolo Guizilini, Rares Andrei Ambrus, Greg Shakhnarovich, Matthew R. Walter Abstract Reconstructing articulated objects prevalent in daily environments is crucial for applications in augmented/virtual reality and robotics. However, existing methods face scalability limitations (requiring 3D supervision or costly annotations), robustness issues (being susceptible to local optima), and rendering shortcomings (lacking speed or photorealism). We introduce SplArt , a self-supervised, category-agnostic framework that leverages 3D Gaussian Splatting (3DGS) to reconstruct articulated objects and infer kinematics from two sets of posed RGB images captured at different articulation states, enabling real-time photorealistic rendering for novel viewpoints and articulations. SplArt augments 3DGS with a differentiable mobility parameter per Gaussian, achieving refined part segmentation. A multi-stage optimization strategy is employed to progressively handle reconstruction, part segmentation, and articulation estimation, significantly enhancing robustness and accuracy. SplArt exploits geometric self-supervision, effectively addressing challenging scenarios without requiring 3D annotations or category-specific priors. Evaluations on established and newly proposed benchmarks, along with applications to real-world scenarios using a handheld RGB camera, demonstrate SplArt ’s state-of-the-art performance and real-world practicality. Code is publicly available at https://github.com/ripl/splart . Figure 1 : Given (top) RGB images of an object at two articulation states, (middle) SplArt uses 3DGS to simultaneously reconstruct its static and dynamic parts and estimate the kinematic articulation model. SplArt is then able to (bottom) render high-fidelity 3D reconstructions of the object along with part-level segmentations for novel articulation states, allowing for novel view synthesis. 1 Introduction Articulated objects, such as drawers, doors, and scissors, are ubiquitous in our daily lives, yet their dynamic nature poses significant challenges for 3D reconstruction—a critical task for applications in augmented/virtual reality [ 37 , 52 ] , robotics [ 51 , 24 , 5 , 1 , 16 , 58 ] , and computer vision [ 46 , 20 ] . Existing methods for reconstructing articulated objects are hindered by several key limitations: they often require labor-intensive supervision (e.g., part-level segmentation or articulation annotations) [ 29 , 41 , 55 , 57 , 23 ] , they depend on 3D supervision that restricts practical use [ 29 , 41 , 23 , 34 , 61 ] , they produce category-specific models that limit scalability [ 29 , 41 , 55 , 60 ] , or they are not capable of real-time, photorealistic rendering [ 29 , 41 , 55 , 57 , 23 , 61 , 30 , 13 , 60 , 6 ] . To address these challenges, we introduce SplArt , a novel self-supervised and category-agnostic framework that leverages 3D Gaussian Splatting (3DGS) [ 26 ] to reconstruct articulated objects from minimal input—two sets of posed RGB images at distinct articulation states. SplArt reconstructs object parts and infers kinematics, enabling real-time, photorealistic rendering for novel views and articulation states. Central to SplArt is the augmentation of 3DGS [ 26 ] to include a differentiable mobility parameter for each Gaussian, which enables a more refined segmentation of static and mobile parts through gradient-based optimization. This results in enhanced reconstruction quality, while preserving the real-time, photorealistic rendering capabilities of 3DGS—offering a speedup of more than 100\times over methods [ 30 , 6 ] based on neural radiance fields [ 39 ] . To enhance robustness, SplArt employs a multi-stage optimization strategy that decouples the part-level reconstruction and articulation estimation processes. Unlike end-to-end approaches prone to local optima [ 30 ] , SplArt first independently reconstructs each articulation state, then estimates each Gaussian’s mobility parameter for part segmentation, and finally refines both the articulation and mobility estimates jointly. This structured approach ensures stable and accurate convergence, avoiding the stringent initialization requirements of existing methods, thereby providing a practical solution for challenging articulated structures. Building on this foundation, SplArt leverages geometric self-supervision to eliminate the need for manual annotations or 3D supervision. By enforcing geometric consistency between reconstructions through complementary loss formulations, SplArt robustly estimates articulation parameters across diverse scenarios. This self-supervised strategy enhances scalability, enabling SplArt to reconstruct a wide range of articulated objects without relying on prior structural or categorical knowledge. Extensive evaluations on both established and newly introduced benchmarks demonstrate SplArt ’s superior articulation accuracy and reconstruction quality, surpassing state-of-the-art methods without requiring 3D supervision. Real-world experiments further validate its practicality, showcasing successful reconstructions of diverse articulated objects using only a handheld RGB camera. In summary, this work contributes: 1. An extension of 3DGS with a differentiable mobility value per Gaussian that enables precise part segmentation using gradient-based optimization. 2. A multi-stage optimization strategy that decouples reconstruction and articulation estimation, enhancing robustness and accuracy. 3. Complementary formulations of geometric self-supervision for articulation estimation, eliminating the need for 3D supervision or laborious annotations. 4. A challenging dataset and new metrics for comprehensive evaluation of articulated object reconstruction. 2 Related Work 2.1 Data-Driven Articulation Learning Estimating the pose and joint properties of articulated objects is crucial for robot manipulation and interaction [ 19 , 11 , 31 , 10 ] . Recent learning-based methods [ 65 , 15 , 57 , 13 , 22 , 33 , 29 , 9 ] infer articulation properties from point clouds via end-to-end training. For instance, Shape2Motion [ 57 ] analyzes motion parts from a single point cloud in a supervised setting, while ANCSH [ 29 ] performs category-level pose estimation but requires class-specific models. RPM-Net [ 63 ] enhances generalization across categories for part segmentation and kinematic prediction, and DITTO [ 23 ] predicts motion and geometry from 3D point cloud pairs without labels. However, these methods depend on costly 3D supervision and annotations. In contrast, our approach reconstructs accurate 3D geometry and detailed appearance, capturing articulation without 3D supervision or priors. 2.2 Representations for Object Reconstruction Early 3D object reconstruction methods predicted point clouds, voxels, or meshes from partial observations [ 8 , 17 , 3 ] . Recent advances in implicit scene representations [ 45 , 38 , 18 , 66 , 39 , 26 ] enable detailed geometry and appearance reconstruction via differentiable rendering [ 56 , 53 , 44 , 12 ] . While neural fields suffer from slow rendering, 3D Gaussian Splatting (3DGS) [ 26 ] overcomes this by using explicit 3D Gaussians. We leverage 3DGS for self-supervised articulated object reconstruction from posed RGB images, achieving fast, realistic synthesis of novel views and articulations in real time. 2.3 Articulated Object Reconstruction Recent methods leverage differentiable 3D representations [ 45 , 39 , 26 ] to jointly reconstruct articulated objects and infer articulation parameters. Training-based approaches use synthetic 3D data to predict joint parameters and segment parts [ 41 , 23 , 13 , 43 , 14 , 25 , 60 ] . Self-supervised methods optimize shape, appearance, and articulation per scene without extensive training [ 30 , 32 , 27 , 50 , 61 , 34 , 6 ] , with some addressing multi-part objects but requiring known part counts and single-level articulation structures [ 61 , 6 , 34 ] . Other works enhance articulation estimation using large language or vision-language models [ 36 , 28 ] . In contrast, our self-supervised method reconstructs two-part articulated objects from RGB images across articulation

Read the complete text