RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos

Yuxin Yao, Zhi Deng, Junhui Hou · 2025

Read RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos by Yuxin Yao, Zhi Deng, Junhui Hou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16822.txt.

Introduction

Yuxin Yao, Zhi Deng, and Junhui Hou present, in a 2025 paper, a way to rebuild moving jointed objects from ordinary video so they can be posed anew. A waving arm or a walking toy must bend at its joints, not melt. Flat video gives no skeleton to bend.

Their method, RigGS, represents the object as a cloud of soft 3D blobs. A skeleton-aware step deforms the cloud over time and proposes joint nodes. Pruning leaves a sparse skeleton. Learned skin and pose-dependent detail layers then drive the motion. The result supports new viewpoints and fresh poses. Animators and robot researchers both need such editable models from cheap input.

Catalog record

CREATORYuxin Yao, Zhi Deng, Junhui Hou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16822

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos Yuxin Yao, Zhi Deng, Junhui Hou Abstract This paper considers the problem of modeling articulated objects captured in 2D videos to enable novel view synthesis, while also being easily editable, drivable, and re-posable. To tackle this challenging problem, we propose RigGS, a new paradigm that leverages 3D Gaussian representation and skeleton-based motion representation to model dynamic objects without utilizing additional template priors. Specifically, we first propose skeleton-aware node-controlled deformation, which deforms a canonical 3D Gaussian representation over time to initialize the modeling process, producing candidate skeleton nodes that are further simplified into a sparse 3D skeleton according to their motion and semantic information. Subsequently, based on the resulting skeleton, we design learnable skin deformations and pose-dependent detailed deformations, thereby easily deforming the 3D Gaussian representation to generate new actions and render further high-quality images from novel views. Extensive experiments demonstrate that our method can generate realistic new actions easily for objects and achieve high-quality rendering. 1 Introduction Rigging involves extracting an animatable skeleton and binding it to a deformable object, which is crucial in movies, games, and AR/VR. The skeleton, composed of interconnected bones and control joints, offers a structured and semantically rich representation of motion. This process is essential for realistic animation, allowing for natural movement and deformation. It also simplifies tasks such as pose editing, motion interpolation, motion transfer, and dynamic animation creation. Creating semantically plausible rigs automatically is challenging. Some methods use universal skeleton models from extensive datasets, like SMPL Loper et al. 2015 for the human body, MANO Romero et al. 2017 for the human hand, and SMAL Zuffi et al. 2017 for animals. These models are essential for object reconstruction, pose estimation, animation, etc. However, many dynamic objects, such as the human body with accessories, hands with personalized gloves, diverse animal species, deformable toys, or robots, cannot be standardized into a universal skeleton. Therefore, it is highly desired to develop a template-free model that can create a rig for any object with a skeletal structure. Some methods extract skeletons from a 3D representation of the object Xu et al. 2019 ; Xu et al. 2020 . However, they rely on artist-designed skeletons as supervision and can only handle symmetric objects. Optimization-based methods Lin et al. 2021 ; Dou et al. 2022 ; Wang et al. 2024 can extract the skeleton from any 3D model, but the skeletons obtained are often very dense. Other approaches combine motion information to extract motion-aware skeletons from point cloud sequences Xu et al. 2022 ; Bae et al. 2022 . However, due to limited 3D data availability, these methods often lack practicality. With the rise of neural rendering, some methods Wu et al. 2023 ; Yang et al. 2023b utilize 2D images or videos to obtain rigs for objects. However, they often require a predefined skeleton structure and only optimize joint positions. Other methods Uzolas et al. 2024 ; Kuai et al. 2023 do not need a predefined skeleton tree but rely on existing techniques to extract skeletons from 3D models, making them dependent on the quality of the reconstructed geometry and skeleton extraction. To address these challenges, we propose RigGS, a template-free automated rigging model that can extract 3D skeletons from monocular videos of moving objects and bind them to drive deformation. Firstly, we utilize a canonical 3D Gaussian shape representation that is deformed by a skeleton-aware node-controlled deformation field over time to initialize the modeling process, producing a set of skeleton-aware nodes. We then simplify the dense nodes using geometric, semantic, and motion information to achieve the final sparse 3D skeleton. Finally, we bind the canonical 3D Gaussians to the 3D skeleton to create a skeleton-driven dynamic model. Once trained, RigGS enables editing, motion interpolation, motion transfer, and animation of dynamic objects while supporting high-quality real-time rendering. Extensive experiments indicate that our RigGS can extract reasonable 3D skeletons, achieves rendering accuracy comparable to state-of-the-art methods, and allows for the flexible generation of new motions for the reconstructed dynamic objects. In summary, we propose a novel template-free paradigm that can synthesize dynamic objects captured in 2D videos from novel views and facilitate editing to create new actions. The technical novelty of our approach lies in: • we introduce a deformation field based on skeleton-aware nodes, combined with 3D Gaussians as the canonical shape representation, achieving simultaneous reconstruction of dynamic objects and obtaining candidate skeleton points; • we propose a heuristic 3D skeleton construction algorithm that considers geometric, semantic, and motion information; and • we develop a skeleton-driven dynamic model with learnable skinning weights to bind the skeleton with the 3D Gaussian representation and a pose-dependent detailed deformation, allowing for flexible generation of new motions. 2 Related work Geometric Representation and Rendering. Traditional 3D modeling utilizes textured meshes as a typical representation. With the emergence of neural implicit representations, many methods are employing them to model 3D objects or scenes Mildenhall et al. 2020 ; Wang et al. 2021 . Recently, 3D Gaussian Splatting (3DGS) Kerbl et al. 2023b has gained attention for its efficient and high-quality rendering capabilities, along with its ease of editing, gradually becoming an important representation in 3D vision. Numerous approaches Wu et al. 2024 ; Huang et al. 2024 ; Yang et al. 2024a ; You and Hou 2024 ; Liu et al. 2025b ; You et al. 2025 have also emerged to address the task of novel view synthesis for dynamic objects or scenes captured by videos. Prior-dependent Dynamic Modeling. Considering the challenges in skeleton extraction, various methods have leveraged category priors to establish specific parameterized models. For example, SMPL Loper et al. 2015 is designed to represent human bodies, MANO Romero et al. 2017 focuses on hand modeling, SMAL Zuffi et al. 2017 extends to handle quadrupeds, and so on. These parameterized models have played important roles in tasks such as reconstruction, tracking, and animation within their domains Jiang et al. 2022 ; Models 2024 ; Moon et al. 2024 ; Yang et al. 2024b . However, their representational capabilities are confined when dealing with shapes beyond their intended distributions, such as humans in intricate attire or animals from diverse categories. To enable broader applications, some methods do not utilize these parameterized models. CASA Wu* et al. 2022 establishes a database of animals, enabling category-agnostic skeletal animal reconstruction from monocular videos by retrieving and refining similar 3D models and skeletons from the database. Some methods take a pre-established skeletal tree structure as input, optimizing the positions of skeletal joints to model objects Yao et al. 2022 ; Wu et al. 2023 ; Yang et al. 2023a . However, the pre-definition of the skeletal tree structure hampers the automation of skeleton creation. Neural Bones for Dynamic Objects. To establish a deformation field for arbitrary objects, BANMo Yang et al. 2022 introduces a representation using neural bones. It no longer relies on the traditional skeletal tree structure; instead, it employs learnable bones, involving spatial positions and transformations, to represent the deformation. These neural bones can be distributed on the surface, inside, or even outside of the object. Similar to some control point-based deformation fields Božič et al. 2021 ; Huang et al. 2024 ;

Read the complete text