Semantic Compression of 3D Objects for Open and Collaborative Virtual Worlds
Jordan Dotzel *, Tony Montes *, Mohamed S. Abdelfattah, Zhiru Zhang · 2025
Read Semantic Compression of 3D Objects for Open and Collaborative Virtual Worlds by Jordan Dotzel *, Tony Montes *, Mohamed S. Abdelfattah, Zhiru Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16679.txt.
Introduction
Virtual worlds need to share 3D objects across weak connections, and squeezing a model the usual way falls apart past about tenfold. Jordan Dotzel and co-authors try a different route in this 2025 paper: semantic compression, which stores what an object is rather than its exact geometry.
Their pipeline writes the object down in plain language, then rebuilds a look-alike with modern generative models, guessing back size, pose, and placement. On a large public object set they reached compression above one hundredfold. In that extreme range their method kept quality better than classic tools, a fit for collaborative virtual spaces.
Catalog record
| CREATOR | Jordan Dotzel *, Tony Montes *, Mohamed S. Abdelfattah, Zhiru Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16679
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic Compression of 3D Objects for Open and Collaborative Virtual Worlds Jordan Dotzel *, Tony Montes *, Mohamed S. Abdelfattah, Zhiru Zhang Abstract Traditional methods for 3D object compression operate only on structural information within the object vertices, polygons, and textures. These methods are effective at compression rates up to 10 × 10\times for standard object sizes but quickly deteriorate at higher compression rates with texture artifacts, low-polygon counts, and mesh gaps. In contrast, semantic compression ignores structural information and operates directly on the core concepts to push to extreme levels of compression. In addition, it uses natural language as its storage format, which makes it natively human-readable and a natural fit for emerging applications built around large-scale, collaborative projects within augmented and virtual reality. It deprioritizes structural information like location, size, and orientation and predicts the missing information with state-of-the-art deep generative models. In this work, we construct a pipeline for 3D semantic compression from public generative models and explore the quality-compression frontier for 3D object compression. We apply this pipeline to achieve rates as high as 10 5 × 10^{5}\times for 3D objects taken from the Objaverse dataset and show that semantic compression can outperform traditional methods in the important quality-preserving region around 100 × 100\times compression. 1 1 footnotetext: Equal contribution 1 Introduction Figure 1 : Semantic Compression: Extreme compression requires primarily storing semantic as opposed to structural information, such as point clouds, polygons, or frequencies. Semantic compression enables extreme ratios by preserving human-oriented semantics as opposed to structural information or its derivatives. In particular, the structured semantic compression outperforms traditional methods in the region around 10-100\times compression, beyond which only semantic-based methods can be applied. The growing popularity of augmented and virtual reality platforms are set to enable near limitless virtual worlds where users can manipulate an ever-increasing number of objects. To make this possible, devices will need large resources to store textures, 3D assets, and animations that must be transferred over the network and stored locally on device. This requires new methods for compression and decompression algorithms that can handle the growing amount of required storage. Semantic compression has been proposed as a new paradigm for storing extremely compressed data [ 20 ] . It focuses on preserving only the human-centric, semantic information in the input by disregarding precise, structural information. These semantics are typically represented with natural language descriptions, which have co-evolved with human society to efficiently describe the most important concepts. These descriptions lead to extremely small representations at the cost of lost structural information such as pixels or polygons. The quality-compression tradeoffs are summarized in Figure 1 , which divides the compression landscape into structural, semantic, and structured semantic regions. Most traditional methods are found within the structural region, such as JPEG for images, MPEG for videos, and decimated meshes for 3D objects. This work, on the other hand, introduces methods for structured semantic compression using edge maps and pure semantic compression using natural language. It further claims that optimal compression with respect to human preferences must transition between the structural and semantic regions, and that this central region outperforms traditional structured compression for most objects. Overall, semantic compression gives rise to two primary advantages over traditional compression: orders of magnitude more objects and storage natively within a human-readable format. This opens up a new set of applications, from augmented and virtual reality, to direct and private storage within brain-computer interfaces. It enables shared virtual worlds with millions of objects that can be collaboratively edited by non-experts. Since the storage format is human-readable, it is portable across many systems and provides the most general interface to end-users. Figure 2 : Semantic Scaling: As the size of the input grows, with more objects and more complex objects (higher resolution, denser point clouds, or more polygons), semantic compression becomes more efficient. Traditional compression scales linearly with the input, while semantic compression scales sub-linearly with the semantic content. The initial memory for the compressed world model is amortized over many objects. Another key advantage is its scalability with input data size, as shown in Figure 2 . Traditional methods scale linearly with the input structural data, like resolution or polygon count, while semantic compression scales with the semantic complexity, such as the number of independent concepts in the input. With most data, the semantic complexity is sub-linear in the input size since most of the structural resolution is redundant and can be regenerated with trained models. It also has privacy implications since certain sensitive features can be removed or obfuscated in storage, such as facial features or other personally identifying information. Despite these advantages, semantic compression comes with additional challenges, especially related to increased computations. It shifts the burden from memory and storage to additional computations that are required to regenerate 3D objects during decompression. With current technology, this requires large multi-modal language models that require billions of floating-point operations. Furthermore, the language model itself can consume a lot of memory, therefore, semantic compression must be used with tens or hundreds of objects to amortize this cost over a large number of stored objects. This, however, fits well with emerging applications in the metaverse which contain vast numbers of objects. In this work, we explore the quality-compression tradeoffs for traditional, structured semantic, and semantic compression of 3D objects, discuss the high-level advantages of semantic compression, and highlight the current technological limitations to guide future work. We summarize our contributions as follows: 1. Construct the first semantic compression pipeline for 3D objects that outperforms traditional methods in the region beyond 10\times compression in terms of human-evaluated quality and compression rate. 2. Explore the quality-performance trade-offs of extreme semantic compression with and without structural information for 3D objects. 3. Compare automated metrics with human evaluations for 3D reconstructions and highlight the limitations of F-Score and CLIP with semantic compression. 2 Background Figure 3 : Semantic Factorization: Across most media formats, the structural and semantic dimensions can be factorized and compressed separately. This stores 3D point clouds or 2D projections in addition to natural language description. Traditional compression methods have achieved impressive compression rates by removing human-imperceptible information. These perceptual compression methods, however, are limited by operating within the structural input space (e.g., pixels or voxels) or direct transformations of them (frequencies or polygons). JPEG, for instance, operates within the frequency domain and removes the least perceptible frequencies according to empirical quantization tables. Semantic compression instead functions within the semantic domain and maps out directions that minimize the perceptual differences. It can also be combined with light-weight structural hints such as sparse point clouds or 2D edge maps to form a hybrid structured semantic approach. This approach attempts to operate on the factorized space of semantic and structural information and compress them separately, as shown in Figure 3 for 3D objects. The structural information, as measured in bits, increases from polygons, to point clouds, to colored voxels, while the semantic information progressively includes more detailed concepts. These two dimensions, while not being completely orthogonal, align well with human preferences, preferring structural fidelity for low compression rates and semantic quality at high compression rates. 2.1 Generative Models Figure 4 : 3D Objects: 3D objects are composed of files for