Text-to-Scene with Large Reasoning Models
Frédéric Berdoz, Luca A. Lanzendörfer, Nick Tuninga, Roger Wattenhofer · 2025
Read Text-to-Scene with Large Reasoning Models by Frédéric Berdoz, Luca A. Lanzendörfer, Nick Tuninga, Roger Wattenhofer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26091.txt.
Introduction
Frederic Berdoz and co-authors build indoor scenes from text in 2025 with help from large reasoning models. A user types a room sketch in words, and the system must fetch the right objects and place them well. Past tools fumbled complex shapes, moves, and long instructions, the team notes.
Their system, Reason-3D, picks objects by captions that cover look, use, and setting. It then places them under both stated and implied rules, and refines spots with collision-aware checks. On tasks from simple to complex, human raters scored it above past methods on look, rule fit, and object choice, they report. The code is shared in public. The work also shows off how well today's reasoning models handle space.
Catalog record
| CREATOR | Frédéric Berdoz, Luca A. Lanzendörfer, Nick Tuninga, Roger Wattenhofer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26091
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Text-to-Scene with Large Reasoning Models Frédéric Berdoz, Luca A. Lanzendörfer, Nick Tuninga, Roger Wattenhofer Abstract Prompt-driven scene synthesis allows users to generate complete 3D environments from textual descriptions. Current text-to-scene methods often struggle with complex geometries and object transformations, and tend to show weak adherence to complex instructions. We address these limitations by introducing Reason-3D, a text-to-scene model powered by large reasoning models (LRMs). Reason-3D integrates object retrieval using captions covering physical, functional, and contextual attributes. Reason-3D then places the selected objects based on implicit and explicit layout constraints, and refines their positions with collision-aware spatial reasoning. Evaluated on instructions ranging from simple to complex indoor configurations, Reason-3D significantly outperforms previous methods in human-rated visual fidelity, adherence to constraints, and asset retrieval quality. Beyond its contribution to the field of text-to-scene generation, our work showcases the advanced spatial reasoning abilities of modern LRMs. Additionally, we release the codebase to further the research in object retrieval and placement with LRMs. 1 1 1 Demo & code: https://lucala.github.io/reason-3d-demo/ Figure 1: Showcase comparison for object retrieval and placement between Reason-3D and baseline approaches for the instruction “A cozy living room of size 5 by 5 units. There is a plant on a small table in front of the L-shaped sofa.” 1 Introduction The demand for customizable 3D scenes is rapidly growing across a wide range of fields, including interior design ( Çelen et al. 2024 ; Fu et al. 2021 ; Dai et al. 2017 ) , video game development ( Raistrick et al. 2024 ; Yang et al. 2024c ; Unity Technologies 2025 ) , autonomous driving ( Gao et al. ; Mao et al. 2024 ) , robotics ( Lee, Park, and Kim 2025 ; Bu et al. 2024 ; Po and Wetzstein 2024 ; Cen et al. 2024 ) , embodied AI ( Yang et al. 2024a ; Lee, Park, and Kim 2025 ; Deitke et al. 2022 ; Yang et al. 2024c ) , and more ( Wen et al. 2025 ) . Generating plausible and physically‐feasible arrangements of objects from high- to low-level descriptions is therefore at the core of text-to-scene models. Traditionally, scene synthesis has been addressed by specialized models trained on annotated layout datasets or by engineered pipelines that encode spatial priors ( Paschalidou et al. 2021 ; Tang et al. 2024 ) . But these methods tend to constrain the ability of the model in some way, be it context confinement, visual fidelity, physical plausibility, or object transformation freedom. Recent advances in training strategies, such as Group Relative Policy Optimization (GRPO), have led to the emergence of Large Reasoning Models (LRMs) ( Shao et al. 2024 ) . Unlike standard LLMs, LRMs are trained to perform multi-step reasoning by leveraging test-time compute through long reasoning traces, enabling them to reason about geometry, context, physical affordances, and object-function relationships ( Xu et al. 2025 ) . This shift enables context-free scene synthesis, where systems can generate and arrange 3D environments directly from natural language, without relying on predefined templates or training distributions. In this work, we present Reason-3D, a modular and highly flexible scene synthesis pipeline that leverages LRMs to construct indoor and outdoor 3D scenes from open-ended textual descriptions. Unlike prior approaches, Reason-3D requires no domain-specific training or architectural constraints. Given a scene prompt, our system extracts relevant objects using a combination of embedding-based retrieval and LLM-based semantic voting across three dimensions: physical, functional, and contextual relevance. We use a dual-stage placement process, where each object is placed in an optimal order autoregressively with the help of an LRM. Finally, Reason-3D refines outputs by making the LRM aware of potential collisions and resolving them. This architecture enables Reason-3D to operate entirely through natural language instructions. It supports complex spatial compositions, unusual object configurations, and even outdoor or hybrid environments without requiring any manual scripting or handcrafted scene rules. We evaluate Reason-3D on established indoor scene synthesis tasks where we significantly outperform prior methods in terms of plausibility and structural coherence (cf. Fig. 1 ). Furthermore, we highlight the generalization capabilities of Reason-3D by demonstrating applications in outdoor scenes and composition scenarios. Through an ablation study, we find that certain LRMs consistently outperform others across a diverse set of spatial reasoning tasks. We summarize our contributions as follows: • We introduce Reason-3D, a novel text-to-scene pipeline that leverages Large Reasoning Models (LRMs) to retrieve and place objects in 3D space, based only on natural language descriptions of the scene, without requiring fine-tuning or pretraining. • Reason-3D significantly outperforms previous baselines by using a dual-phase placement strategy combining autoregressive layout with collision-aware refinement, enabling physically coherent scene synthesis. • Reason-3D generalizes to any scene setting, as demonstrated by our outdoor scene synthesis and open-world compositions. By being able to run out-of-the-box on any object library without requiring scripting or hand-crafted rules. Figure 2: Overview of our proposed architecture . Retrieval : We start by processing objects from an asset library, generating images of these objects for captions and orientation. A captioning model creates object descriptions, which are then turned into embedding vectors and stored in a vector database. Given an instruction, we extract a list of objects that would be feasible according to the instruction. and subsequently query the database for such objects. Placement : Given the instruction and retrieved objects, we extract a set of constraints to determine an ordered sequence for object placement. Once all objects are placed, the scene is refined by calculating and adjusting for object collisions. Figure 3: Qualitative comparison for object retrieval and placement across various scenes. We find that overall, Reason-3D can better follow instructions and place objects reasonably. Compared to Holodeck and Reason-3D, LayoutVLM was not designed to retrieve objects. We use the objects retrieved from Reason-3D for LayoutVLM. Figure 4: Qualitative comparison of object placement performance when instruction complexity is increased. Every scene is generated from scratch with the entire instruction given up to and including its circled number. 2 Related Work Recent progress in indoor scene synthesis has primarily followed two distinct approaches. One line of work capitalizes on the strong generative capabilities of image-based models, often employing Neural Radiance Fields (NeRFs) or 3D Gaussian splats as output representations ( Schult et al. 2024 ; Zhou et al. 2024 ; Epstein et al. 2024 ; Po and Wetzstein 2024 ) . While these methods produce visually realistic results, the generated scenes lack object-level separability, making them unsuitable for downstream tasks requiring precise object manipulation or interaction. A second research direction focuses on structured scene generation using intermediate representations, such as scene graphs or layout templates, coupled with curated asset libraries to synthesize 3D environments with discrete objects ( Feng et al. 2023 ; Yang et al. 2024c ) . Neural-3D generation. Within this structured generation paradigm, learning-based generative models such as diffusion models have emerged as powerful tools for modeling spatial priors ( Hu et al. 2024 ) . DiffuScene ( Tang et al. 2024 ) applies a denoising diffusion process to generate unordered sets of object attributes, synthesizing layouts from the 3D-FRONT dataset, which comprises 19,000 annotated indoor scenes ( Fu et al. 2021 ) . Other methods use model training with scene graphs ( Yang