TRELLISWorld: Training-Free World Generation from Object Generators

Hanke Chen Yuan Liu Minchen Li · 2025

Read TRELLISWorld: Training-Free World Generation from Object Generators by Hanke Chen Yuan Liu Minchen Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23880.txt.

Introduction

Hanke Chen and two co-authors present TRELLISWorld, a training-free way to generate 3D scenes from text by reusing object generators. Text-to-3D tools usually make single objects, need special training for scenes, or cannot be viewed from all sides. The authors instead treat a scene as a set of overlapping 3D tiles, each produced by a text-to-3D diffusion model meant for objects.

The tiles are denoised separately and then blended with weighted averaging where they overlap, so seams fade and the scene stays coherent. No scene-level dataset or retraining is required, and minimal heuristics glue the process together. Because the method borrows object-level priors wholesale, it inherits the ability to generalize to new prompts. The team shows layouts, efficient generation, and easy editing. They offer the approach as a foundation for language-driven scene building in prototyping, virtual reality, and simulation.

Catalog record

CREATORHanke Chen Yuan Liu Minchen Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-23880

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TRELLISWorld: Training-Free World Generation from Object Generators Hanke Chen Yuan Liu Minchen Li Abstract Text-driven 3D scene generation holds promise for a wide range of applications, from virtual prototyping to AR/VR and simulation. However, existing methods are often constrained to single-object generation, require domain-specific training, or lack support for full 360-degree viewability. In this work, we present a training-free approach to 3D scene synthesis by repurposing general-purpose text-to-3D object diffusion models as modular tile generators. We reformulate scene generation as a multi-tile denoising problem, where overlapping 3D regions are independently generated and seamlessly blended via weighted averaging. This enables scalable synthesis of large, coherent scenes while preserving local semantic control. Our method eliminates the need for scene-level datasets or retraining, relies on minimal heuristics, and inherits the generalization capabilities of object-level priors. We demonstrate that our approach supports diverse scene layouts, efficient generation, and flexible editing, establishing a simple yet powerful foundation for general-purpose, language-driven 3D scene construction. We will release the full implementation upon publication. \svgpath svg/ \svgsetup inkscapelatex=true, inkscapeversion=auto Figure 1: Scenes generated by our framework, TRELLISWorld , using only natural language input. Users may provide fine-grained prompts for specific regions, enabling semantically consistent gradual transitions: e.g., from a dense commercial district with greens, into low-density residential zones. 1 Introduction Generating 3D worlds from text represents a longstanding goal at the intersection of computer graphics, machine learning, and human-computer interaction. Enabling users to describe a virtual world in natural language and synthesize an editable 3D environment would transform multiple domains. For example, in creative content design, this could accelerate ideation workflows, allowing a game designer to rapidly prototype a level using coarse spatial prompts, or enabling a video creator to synthesize a virtual scene as a storytelling background. Recent advances in 3D content generation, ranging from SDS-based methods distilling 2D priors into neural fields ( Poole et al. 2022 ; Wang et al. 2023 ; Wang et al. 2022 ; Tang et al. 2024b ) , to multi-view diffusion for geometric consistency ( Liu et al. 2023b ; Shi et al. 2023 ; Shi et al. 2024 ; Liu et al. 2024 ) , and emerging 3D-native models like DiTs ( Peebles & Xie 2023 ; Hong et al. 2024b ; Lai et al. 2025 ; Zhang et al. 2024b ) , have significantly improved 3D object synthesis. However, these methods are predominantly limited to single-object generation, with limited progress toward generating entire 3D scenes. While directly generating 3D scenes is an active research direction, current scene-generation models tend to be either (1) domain-specific, trained on narrow datasets like indoor rooms or driving scenes; or (2) produce image-based representations such as panoramic or spherical projections, which are not designed for full 360-degree spatial interaction or view synthesis. This limitation arises from limited large-scale, diverse, general-purpose datasets for 3D scenes comparable to those available in 2D vision or object-centric 3D generation. Scenes contain long-range dependencies, heterogeneous object types, and complex spatial arrangements that are difficult to annotate and curate. Therefore, a method that can leverage object-centric generation to generate scenes is highly desired. Recently, SynCity ( Engstler et al. 2025 ) demonstrates promising results in adapting object generators for city-scale scene generation. However, its dependence on 2D inpainting ( Lugmayr et al. 2022 ) introduces a fundamental limitation. Errors in the image domain can propagate and destabilize the 3D reconstruction. This raises a natural question: can we instead generate scenes directly in 3D space, bypassing the need for 2D intermediate representations? In this work, we introduce TRELLISWorld , a training-free approach to text-driven 3D scene generation by leveraging general-purpose text-to-3D object diffusion models for scene composition. Our key insight is to reformulate global 3D scene synthesis as a multi-tile denoising problem, wherein the scene is partitioned into spatially overlapping regions that are independently denoised and later blended using a weighted averaging scheme in one diffusion step. This formulation offers a practical and scalable alternative to end-to-end scene-level training, enabling high-quality scene generation at significantly reduced cost. We will release the full implementation upon publication. Our method offers several advantages: • Training-free and editable : Taking advantages of 3D environments’ multi-scale signal structure, our approach requires no scene-level dataset or fine-tuning. It inherits editability and generalization capabilities from the underlying object-level generator, e.g., TRELLIS ( Xiang et al. 2025 ) . • Simple and general : Our method requires minimal task-specific heuristics, making it broadly applicable across diverse scene types. • Scalable and smooth : Compared to prior methods, our tile-wise approach is computationally efficient, blends overlapping regions smoothly, and enables generations of significantly larger and more coherent scenes. 2 Related Work Figure 2: Illustration of our tiled diffusion process. We first split the scene noise into multiple tiles to denoise each tile in parallel. Then we take the weighted average described in Equation 1 for each tile and aggregate the result to obtain the scene noise for previous timesteps. This process is detailed by Equation 2 . 2.1 Foundation of Reconstruction and Object Generation Benefiting from recent advances in 3D representations for reconstruction, such as NeRF ( Mildenhall et al. 2020 ; Yu et al. 2021 ; Müller et al. 2022 ; Shue et al. 2022 ; Chen et al. 2022 ; Kerbl et al. 2023 ; Zhang et al. 2020 ; Barron et al. 2021 ; Barron et al. 2022 ; Barron et al. 2023 ) , Score Distillation Sampling (SDS)-based methods ( Poole et al. 2022 ; Wang et al. 2023 ; Wang et al. 2022 ; Tang et al. 2024b ) distill the knowledge of 2D diffusion models into the creation of 3D objects using differentiable rendering. Specifically, NeRF++ ( Zhang et al. 2020 ) , combined with ProlificDreamer ( Wang et al. 2023 ) , demonstrates the first possibility of generating 3D scenes by distilling 2D diffusion with an added density prior. Later, multi-view diffusion methods ( Liu et al. 2023b ; Shi et al. 2023 ; Shi et al. 2024 ; Liu et al. 2024 ) were proposed to address the Janus (multi-face) problem. However, as SDS depends on test-time optimization, object generation can take up to 40 minutes. To overcome this, methods like LRM have been proposed to generate 3D representations directly from images ( Liu et al. 2023a ; Hong et al. 2024b ; Gao et al. 2024 ; Lai et al. 2025 ) and texts ( Jun & Nichol 2023 ; Hong et al. 2024a ; Zhang et al. 2024b ) . Although none of these works aim to generate large-scale scenes, their technologies lay the foundation for 3D scene generation. 2.2 Scene Generation Based on 2D Generation While SDS-based methods above often focus on novel view synthesis or scene reconstruction, another line of work leverages the capabilities of 2D diffusion models without relying on test-time optimization. Early works extend a single image autoregressively into a video sequence, guided by camera trajectories ( Liu et al. 2021 ; Li et al. 2022 ; Chai et al. 2023 ; Cai et al. 2023 ) . These methods often project depth-estimated results into point clouds and fill in the missing regions under different camera extrinsics. While initial work focused mainly on natural scenes,

Read the complete text