Utonia: Toward One Encoder for All Point Clouds

2026

Read Utonia: Toward One Encoder for All Point Clouds (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03283.txt.

Introduction

Point clouds are sets of points that record the shape of a scene in three dimensions, but each sensor makes them differently. Yujia Zhang and colleagues present Utonia in 2026, a single self-supervised encoder meant to learn from all of them at once. Training mixes remote sensing scans, outdoor lidar, indoor depth sequences, computer-designed object models, and clouds lifted from plain videos.

Trained together, these sources produce one shared representation space that transfers across domains, and new behaviors appear that single-domain training misses. The features also help beyond perception. Robot control policies using them handle manipulation better, and vision-language models using them reason better about space. The authors frame Utonia as an early step toward foundation models for sparse three-dimensional data.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53386
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03283

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Utonia: Toward One Encoder for All Point Clouds Abstract We dream of a future where point clouds from all domains can come together to shape a single model that benefits them all. Toward this goal, we present Utonia, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos. Despite their distinct sensing geometries, densities, and priors, Utonia learns a consistent representation space that transfers across domains. This unification improves perception capability while revealing intriguing emergent behaviors that arise only when domains are trained jointly. Beyond perception, we observe that Utonia representations can also benefit embodied and multimodal reasoning: conditioning vision-language-action policies on Utonia features improves robotic manipulation, and integrating them into vision-language models yields gains on spatial reasoning. We hope Utonia can serve as a step toward foundation models for sparse 3D data, and support downstream applications in AR/VR, robotics, and autonomous driving. Yujia Zhang 1 , Xiaoyang Wu 1† , Yunhan Yang 1 , Xianzhe Fan 1 , Han Li 3 , Yuechen Zhang 2 , Zehao Huang 3 , Naiyan Wang 3 , Hengshuang Zhao 1‡ 1 The University of Hong Kong 2 The Chinese University of Hong Kong 3 Xiaomi † Project lead ‡ Corresponding author https://pointcept.github.io/Utonia Figure 1 : One encoder for all point clouds. We visualize Utonia features by PCA. City-scale geometry with large coordinate ranges: the smooth, coherent features over terrain and building masses indicate robustness to an extreme extent; Outdoor LiDAR with sparse ring-like scan patterns: features remain organized by geometry rather than scanlines, suggesting reduced reliance on sampling-specific shortcuts. Indoor reconstruction in dense scans: features separate in object level, reflecting geometry-aware scene structure. Object-centric scans with orientation invariance: features exhibit consistent part-level regions with orientation-robust semantics. Despite differences in extent, density, patterns, and coordinate conventions, the representation remains structured and semantically meaningful. 1 Introduction Spatial cognition is drawing increasing attention, driven by applications in autonomous driving ( 31 ; 5 ) , AR/VR ( 3 ) , and robotics ( 12 ) . Today’s pipelines remain dominated by dense images and videos ( 46 ; 47 ) . While effective, these dense formats are redundant carriers of 3D information and do not explicitly enforce geometric consistency. Instead, sparse point clouds provide a geometry-explicit compression of the physical world, yielding a compact representation that complements or conditions dense-centric models. Yet learning from sparse point clouds is inherently harder than learning from images. Images live on a standard dense grid, whereas point clouds are sparse and unstructured samples. Their scale, density, sampling pattern, and auxiliary modalities vary with sensors and preprocessing, leading to extreme domain shifts. For example, outdoor LiDAR scenes are sparse, span large extents with characteristic scan-line patterns, and often lack color or normal, while object-centric scans are denser and in much smaller ranges. Under such variation, a single pipeline can be dominated by domain-specific priors: scales, sparse patterns, or modality availability, rather than transferable geometry or semantics, causing representations to cluster by domain. Table 1 : Desirable properties of powerful point cloud models: Sonata scales up point SSL within indoor/outdoor (In./Out.) domain separately. Concerto inherits this recipe and further scales pretraining with video-lifted point clouds. Utonia steps toward one encoder for all these data with additional object assets, supporting optional color/normal input. C./N. indicates how each model handles inconsistent presence of color/normal. Models In. Out. Obj. Video C./N. PointMAE ( 24 ) ✗ ✗ ✓ ✗ w/o PointM2AE ( 53 ) ✗ ✗ ✓ ✗ w/o MSC ( 44 ) ✓ ✗ ✗ ✗ w/ Sonata ( 40 ) ✓ 1 ✓ 2 ✗ ✗ w/ Concerto ( 55 ) ✓ 1 ✓ 2 ✗ ✓ 1 w/ Utonia ✓ ✓ ✓ ✓ adaptive Figure 2 : Cross-domain semantic similarity. Human perception operates at a fixed angular resolution, resulting in similar perception granularity between a close small toy car and a far-away real car, which motivates semantic matching at a canonical granularity across domains. Utonia representations exhibit high similarity between the toy car from object CAD and real cars in outdoor scenes, while the previous SOTA Concerto fails to align. Figure 3 : Gravity priors influence. Scene-level data have a strong z-axis up prior. Utonia steps further to erase such assumptions by including rotation-invariant objects with strong SE(3) augmentations into pretraining datasets. Thus, unlike dense images, universality does not arise in point cloud self-supervised learning (SSL) by simply mixing datasets. Point cloud SSL remains domain-fragmented due to practical design choices rather than the structure of the physical world. Specifically, prior methods, Sonata ( 40 ) and Concerto ( 55 ) , are typically trained within a single domain and transfer poorly across domains, even though all point clouds are discrete observations of the same 3D reality. As shown in Figure 2 , when querying the front of a toy car from the object CAD and retrieving it by feature similarity in an outdoor LiDAR scene, Concerto struggles to highlight the corresponding region on the real car. However, recent progress has matured both ingredients needed for unification: scalable point backbones and robust SSL recipes. PTv3 ( 41 ) offers a strong foundation, while Sonata ( 40 ) and Concerto ( 55 ) mitigate geometric shortcuts via perturbations and cross-modal synergy, making large-scale training stable within individual domains. We argue the next step is to move beyond siloed pretraining and learn a single encoder across domains. It is time to turn fragmented 3D observations into a shared representation. Motivated by this, we take a first step toward an all-for-one, one-for-all point encoder: learned from and shared for diverse point clouds. We present Utonia , not as a definitive solution, but as an exploration that exposes what breaks joint training and distills a minimal set of fixes. Specifically, we identify three core mismatches across domains: non-unified input channels (e.g., optional color/normal/image cues), inconsistent sparsity and density, and perceptual granularity shifts induced by scenes and downstreams. Utonia addresses them with three simple, domain-agnostic designs: Causal Blinding via randomized modality masking (e.g., dropping color/normal) for robustness to missing channels, granularity-prompted rescaling to harmonize effective perception scales, and RoPE-enhanced positional hints for more transferable geometry encoding, while avoiding domain-specific modules. With these minimal fixes, we can stably train a single encoder on 250k cross-domain point clouds, augmented with 1M additional CAD assets, yielding a unified representation space and our cross-domain pretrained model, Utonia. Surprisingly, joint pretraining yields several emergent behaviors. First, object-, indoor-, and outdoor-scale data begin to benefit under a shared encoder, rather than compete across domains. Second, the learned representations retain scene-level gravity alignment while remaining largely gravity-irrelevant for object-centric geometry. Beyond 3D perception, the pretrained encoder benefits spatial reasoning when integrated into VLM backends and improves robotic manipulation when conditioning VLA policies. We hope these findings highlight sparse point clouds as a compelling substrate toward unified spatial cognition, turning fragmented 3D observations into a shared representation. 2 Pilot Study: What Prevents Unification? Table 2 : Scale and grid size . Naive joint pretraining with per-dataset default grid sizes is unstable and causes large performance drops. Grid size jittering offers limited robustness. Fixing

Read the complete text