RoomPlanner: Explicit Layout Planner for Easier LLM-Driven 3D Room Generation
Wenzhuo Sun, Mingjian Liang, Wenxuan Song, Xuelian Cheng, Zongyuan Ge · 2025
Read RoomPlanner: Explicit Layout Planner for Easier LLM-Driven 3D Room Generation by Wenzhuo Sun, Mingjian Liang, Wenxuan Song, Xuelian Cheng, Zongyuan Ge (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17048.txt.
Introduction
Wenzhuo Sun and four co-authors introduce RoomPlanner, a framework that turns a short text prompt into a full three-dimensional room. Earlier systems needed hand-designed layouts or panoramic images to guide them. RoomPlanner instead plans the layout itself, so a brief description is the only input a user must supply.
A chain of language-driven planners first expands the prompt into detailed descriptions of each object and the background, then seeds three-dimensional point clouds. Arrangement rules then shift objects until the layout has no collisions and leaves usable space. A new camera-path sampling method speeds up the final rendering, keeping total generation under half an hour. Experiments show rooms that look realistic, beat prior methods on speed and quality, and stay editable afterward.
Catalog record
| CREATOR | Wenzhuo Sun, Mingjian Liang, Wenxuan Song, Xuelian Cheng, Zongyuan Ge |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-17048
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RoomPlanner: Explicit Layout Planner for Easier LLM-Driven 3D Room Generation Wenzhuo Sun, Mingjian Liang, Wenxuan Song, Xuelian Cheng, Zongyuan Ge Abstract In this paper, we propose RoomPlanner , the first fully automatic 3D room generation framework for painlessly creating realistic indoor scenes with only short text as input. Without any manual layout design or panoramic image guidance, our framework can generate explicit layout criteria for rational spatial placement. We begin by introducing a hierarchical structure of language-driven agent planners that can automatically parse short and ambiguous prompts into detailed scene descriptions. These descriptions include raw spatial and semantic attributes for each object and the background, which are then used to initialize 3D point clouds. To position objects within bounded environments, we implement two arrangement constraints that iteratively optimize spatial arrangements, ensuring a collision-free and accessible layout solution. In the final rendering stage, we propose a novel AnyReach Sampling strategy for camera trajectory, along with the Interval Timestep Flow Sampling (ITFS) strategy, to efficiently optimize the coarse 3D Gaussian scene representation. These approaches help reduce the total generation time to under 30 minutes. Extensive experiments demonstrate that our method can produce geometrically rational 3D indoor scenes, surpassing prior approaches in both rendering speed and visual quality while preserving editability. The code will be available soon. Introduction Text-to-3D scene generation aims to convert textual descriptions into 3D environments. Designed to lower costs and reduce the required expertise, this paradigm is conducive to various downstream applications, such as embodied AI ( 1 ; 34 ) , gaming ( 41 ) , filmmaking, and augmented / virtual reality ( 26 ) . Benefitting from the remarkable success of generation models ( 11 ; 31 ) and implicit neural representation ( 23 ; 15 ) , numerous methods have emerged to address the challenges of text-to-3D object generation, achieving significant milestones. However, directly transitioning from generating 3D assets to creating complete scenes is not a trivial task. This is partly due to the fact that layout information requires additional context and human effort. Based on the extracted layout sources, existing 3D scene generation methods can be broadly classified into two categories: I. Visual-guided , and II. Rule-based methods. Visual-guided methods rely on extracting implicit spatial information from visual representations, e.g., multi-view RGB or panorama images. Methods in this category typically adopt a two-stage generation framework, where image generation serves as an intermediate process, followed by an implicit neural representation network that renders a 3D scene from previously generated images. Some methods ( 18 ; 13 ) treat objects and backgrounds as an indivisible whole as input for the implicit rendering network. As a result, these continuous representations produced by these methods lack object-level decoupling, which significantly restricts practical use. These kinds of approaches can also lead to foggy or distorted geometry, as shown in fig. 1 (b). Instead of being directly constrained, the layout configuration is learned from a text-to-image generation model, which is optimized by aligning the generated images with the input text using a similarity score. Thus, the optimization of the scene configuration is constrained by the capabilities of the image generation model, making it difficult to directly control the layout through textual descriptions. Rule-based methods, on the other hand, explicitly utilize predefined rules based on physical relationships and constraints to create layout information of 3D environments. Early methods ( 17 ; 21 ; 2 ; 12 ) depend on manually crafted prompt templates or human-designed layouts for initialization, which helps generate physically plausible scenes. This procedure relies on specialized expert knowledge and typically involves a tedious trial-and-error process to refine predefined spatial attributes of objects, e.g., position, size, and rotation etc. . This human-in-the-loop paradigm increases the demand for human labor while also undermining the end-to-end training process. Figure 1: Compared to previous methods, ie., visual-guided method Pano2Room ( 30 ) and rule-based method DreamScene ( 17 ) , our approach effectively generates indoor scenes characterized by (a) more realism, (b) smoother mesh structures, and (c) support for a diverse array of editions, including operations such as rotation, translation, importing/deleting 3D assets, and style variations. On a separate line of research and to reduce human effort in layout design, data-driven approaches treat scene synthesis as a 3D asset arrangement task within a bounded environment. These methods add objects sequentially using an order prior with a forest structure ( 32 ) or simultaneously through conditional score evaluation ( 22 ) . Recently, Large Language Models (LLMs) have been introduced to automate the creation of 3D scenes ( 40 ; 39 ; 16 ) . LLMs can directly translate unstructured language into structured spatial representations, which reduces dependence on rigid templates. However, their reliance on existing 3D assets limits the versatility and flexibility required to generate diverse and innovative scenes. In this paper, we present RoomPlanner , which combines the advances of visual-guided and rule-based models. Unlike previous approaches, our method enables automatic generation of differentiable 3D scenes and layout constraints given only short textual prompts. Our framework adopts a ‘Reasoning-Grounding, Arrangement, and Optimization’ pipeline to break down the complex 3D scene generation task into controllable and scalable subtasks. In the reasoning and grounding stage, our hierarchical LLM-driven agent planners can parse short user prompts into executable long-term descriptions. Specifically, our high-level LLM agent produces textual descriptions that select the initial target scene and objects that satisfy the contextual semantics. The low-level LLM agent generates fine-grained grounding descriptions that infer object and background scales, along with layout rules that match real-world dimensions. These explicit physical constraints also ensure that the generated 3D scenes meet the interaction requirements. After obtaining the initial scene description from the LLM agents, we utilize a text-to-3D object generator ( 14 ) to create 3D assets represented as point clouds. To assign initial object assets into the scene, we then employ interactive optimization to update the layout design by verifying whether the generated layout meets our constraints. To improve the rationality of the indoor scene, we first apply collision constraints to detect overlaps between objects. Additionally, we introduce a reachability constraint to assess path viability between entry points and target assets, supporting dynamic asset placement through closed-loop optimization. The generated scenes remain fully editable at both the object and global levels, allowing for reposition, insertion, deletion, and style transfer, as shown in fig. 1 (c). For photorealistic rendering, we introduce a module that integrates Interval Timestep Flow Sampling (ITFS) with reflection flow matching and hierarchical multi-timestep sampling. By leveraging a 2D diffusion prior and detailed scene descriptions from LLM agents, this module generates continuous 3D representations that exhibit a realistic appearance and rich semantics. We also propose the AnyReach camera trajectory sampling strategy that can optimize both object-centric and global views within a single rendering pass. Consequently, our pipeline produces high-fidelity scenes in a single optimization stage, reducing the overall runtime by nearly 2× compared to prior work. Related Work Text-to-3D Object Generation Existing 3D scene generation methods can be broadly classified into two categories: I. Visual-Guided , and II. Direct regression methods. Visual-Guided methods ( 35 ; 19 ; 36 ) rely on pretrained 2D diffusion models, which can lead to error accumulation during the subsequent image-to-3D conversion process.