Cross-Instance Gaussian Splatting Registration via Geometry-Aware Feature-Guided Alignment

Roy Amoyal, Oren Freifeld, Chaim Baskin · 2026

Read Cross-Instance Gaussian Splatting Registration via Geometry-Aware Feature-Guided Alignment by Roy Amoyal, Oren Freifeld, Chaim Baskin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21936.txt.

Introduction

Amoyal and co-authors align two 3D Gaussian splatting models even when they show different objects of the same kind. Prior tools could only register two scans of the very same object, and often needed the true scale given. Their GSA method finds matches with viewpoint-guided spherical map features. A coarse step solves orientation, surviving flips of 180 degrees and tenfold scale gaps. A fine step then enforces feature agreement across views. In same-object tests GSA won by wide margins, and it gives the first category-level alignment.

Catalog record

CREATORRoy Amoyal, Oren Freifeld, Chaim Baskin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58631
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21936

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cross-Instance Gaussian Splatting Registration via Geometry-Aware Feature-Guided Alignment Roy Amoyal, Oren Freifeld, Chaim Baskin Abstract We present Gaussian Splatting Alignment (GSA), a novel method for aligning two independent 3D Gaussian Splatting (3DGS) models via a similarity transformation (rotation, translation, and scale), even when they are of different objects in the same category ( e.g . , different cars). In contrast, existing methods can only align 3DGS models of the same object ( e.g . , the same car) and often must be given true scale as input, while we estimate it successfully. GSA leverages viewpoint-guided spherical map features to obtain robust correspondences and introduces a two-step optimization framework that aligns 3DGS models while keeping them fixed. First, we apply an iterative feature-guided absolute orientation solver as our coarse registration, which is robust to poor initialization ( e.g . , 180° misalignment or a 10× scale gap). Next, we use a fine registration step that enforces multi-view feature consistency, inspired by inverse radiance-field formulations. The first step already achieves state-of-the-art performance, and the second further improves results. In the same-object case, GSA outperforms prior works, often by a large margin, even when the other methods are given the true scale. In the harder case of different objects in the same category, GSA vastly surpasses them, providing the first effective solution for category-level 3DGS registration and unlocking new applications. Project webpage: https://bgu-cs-vil.github.io/GSA-project (A) Cross-instance registration (a) Random Initialization: view 1 (b) GSA (ours) Result: view 1 (c) Random Initialization: view 2 (d) GSA (ours) Result: view 2 (B) Category geometric-consistent object replacement (e) Source Object (f) Intra-dataset Replacement (g) Intra-dataset Replacement (h) Inter-dataset Replacement Figure 1 : (A) Cross-instance registration: GSA successfully aligns the red boat to the white boat despite extremely poor initialization, including a 180° rotation and over 5× scale difference. (B) GSA enables geometrically consistent object replacement even across datasets, e.g., replacing the white car from 3D Real Car Du et al. 2024 with purple/red cars from the same dataset and a police car from ShapeNet Chang et al. 2015 . 1 Introduction 3D Gaussian Splatting (3DGS) Kerbl et al. 2023 is a powerful representation for efficient, high-fidelity novel view synthesis. However, aligning two independent 3DGS models is an open challenge. To date, no dedicated 3DGS registration framework exists, let alone at the category level, where inter-model differences (in shape/scale/appearance) complicate the task. Existing point cloud registration methods Besl and McKay 1992 ; Zhou et al. 2016 ; Yew and Lee 2022 ; Qin et al. 2022 struggle to align 3DGS models, whose point distributions are non-uniform and often have arbitrary scale, position, and orientation as they rely on structure-from-motion (SfM) ( e.g . , COLMAP Schönberger and Frahm 2016 ). We define the 3DGS alignment task as estimating a similarity transformation (rotation, translation, and scale) that aligns one 3DGS model to another, achieving geometric alignment and visual consistency. Although also definable at the scene level, we focus on objects. Moreover, we also target the alignment of different objects from the same category ( e.g . , cars), where intra-class variation in pose, structure, and detail makes the task especially challenging. We introduce G aussian S platting A lignment (GSA) , the first method that solves this task. This enables applications such as synchronized novel view synthesis across instances, facilitating tasks such as dataset visualization and semantic analysis, as well as object replacement ( Fig. 1 , B), where one object replaces another in a geometrically and photometrically consistent manner. This is made possible by precise alignment, and can be seamlessly integrated within 3DGS-based object removal and scene inpainting techniques Wang et al. 2024 ; Chen et al. 2023 ; Huang et al. 2025 ; Kocour et al. 2025 . Existing methods, such as GaussReg ( Chang et al. 2024 ) , rely on Iterative Closest Point (ICP) ( Besl and McKay 1992 ) and struggle to align even models of the same object. With different objects, their performance collapses due to initialization sensitivity, structural variation, and poor correspondences. In contrast, GSA achieves accurate alignment in both cases, outperforming prior methods for same-object registration and yielding the first solution for 3DGS category-level alignment. See Fig. 1 . GSA starts with building two 3DGS models augmented with geometry-aware features that encode spatial and semantic cues for robust correspondence. It then performs coarse registration using a new feature-guided absolute orientation solver to estimate a similarity transformation. Next, it refines the transformation via iterative multi-view feature-consistency optimization. Unlike prior work, GSA leverages feature-driven reasoning throughout, enabling accurate alignment across models even under large rotations (including 180°), scale variation, and structural noise. While several 3DGS-based models Zuo et al. 2024 ; Zhou et al. 2024 ; Qin et al. 2024 ; Marrie et al. 2024 also lift features ( e.g . , DINOv2 ( Oquab et al. 2023 ) ) from 2D to 3D, these features are too ambiguous (as we show) for the alignment task. Thus, when we build the 3DGS models we opt to lift a different type of semantic features, that is geometry-aware Mariotti et al. 2024 . This choice, combined with our method, enables reliable alignment in challenging conditions. Our key contributions are as follows: 1) We extend ICP to an iterative absolute-orientation solver that robustly converges even under poor initializations ( e.g . , 180° rotations) and large scale mismatches ( e.g . , 10×). It remains accurate even in cross-instance registration, despite geometric variation, by introducing semantic-geometric feature constraints that guide correspondences beyond pure geometry. 2) A novel inverse-radiance-field formulation for registration, yielding a multi-view feature-field consistency method that attains near-perfect same-object registration and surpasses prior state-of-the-art by a large margin in cross-instance cases. 3) Our method enables novel applications including geometrically-consistent object replacement and synchronized novel-view synthesis for coherent rendering and scene manipulation. 2 Related work Self-Supervised Visual Features and Geometry-Aware Enhancements. Self-supervised methods such as DINO Caron et al. 2021 , DINOv2 Oquab et al. 2023 , and iBOT Zhou et al. 2022 yield robust, generalizable visual features. Zhang et al . Zhang et al. 2023 combine DINO with Stable Diffusion for zero-shot correspondence, yet these features still lack 3D geometric awareness. This motivated recent work to add spatial cues. Zhang et al . Zhang et al. 2024 fine-tuned features for geometry-aware matching but struggled with 3D cross-view symmetries. Mariotti et al . Mariotti et al. 2024 addressed this by adding weak viewpoint supervision into DINOv2 via spherical maps. As part of our method, we adopt the 2D features from Mariotti et al. 2024 to improve 3D alignment. Unlike prior methods lacking geometric context or struggling with spatial ambiguity, our approach extends 2D semantic alignment principles to 3D, enabling spatially-consistent 3DGS registration, even across different objects (within a category). Features in 3DGS. Recent works extend 3DGS with semantic features from 2D models for tasks beyond novel view synthesis. Feature3DGS Zhou et al. 2024 embeds SAM Kirillov et al. 2023 and LSeg Li et al. 2022 via a convolutional decoder for segmentation and editing. FMGS Zuo et al. 2024 integrates CLIP Radford et al. 2021 and DINO with multi-resolution hash encoding for efficient detection and segmentation. LangSplat Qin et al. 2024 compresses CLIP features into a

Read the complete text