RoomCraft: Controllable and Complete 3D Indoor Scene Generation Conference: ; ;
Mengqi Zhou*, Xipeng Wang*, Yuxi Wang † \dagger, Zhaoxiang Zhang † \dagger · 2025
Read RoomCraft: Controllable and Complete 3D Indoor Scene Generation Conference: ; ; by Mengqi Zhou*, Xipeng Wang*, Yuxi Wang † \dagger, Zhaoxiang Zhang † \dagger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22291.txt.
Introduction
Mengqi Zhou and co-authors present a 2025 system that builds full 3D indoor scenes from loose inputs. Users may start from a photo, a sketch, or plain text. A good scene must keep objects apart, respect links between them, and look real. Past neural tools repeat objects, while rule-based tools drop objects when rules clash.
Their system, RoomCraft, works in stages from input to layout. It first pulls room type, objects, and spatial links into a structured form. A network of links then guides the order in which objects are placed. A search method keeps that order coherent. A smart placement step shifts weights to cut collisions and keep all objects. Tests show RoomCraft ahead of past methods on realism and fit. The authors stress control and completeness as joint goals.
Catalog record
| CREATOR | Mengqi Zhou*, Xipeng Wang*, Yuxi Wang † \dagger, Zhaoxiang Zhang † \dagger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-22291
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RoomCraft: Controllable and Complete 3D Indoor Scene Generation Conference: ; ; Mengqi Zhou*, Xipeng Wang*, Yuxi Wang † \dagger, Zhaoxiang Zhang † \dagger Abstract . Generating realistic 3D indoor scenes from user inputs remains a challenging problem in computer vision and graphics, requiring careful balance of geometric consistency, spatial relationships, and visual realism. While neural generation methods often produce repetitive elements due to limited global spatial reasoning, procedural approaches can leverage constraints for controllable generation but struggle with multi-constraint scenarios. When constraints become numerous, object collisions frequently occur, forcing the removal of furniture items and compromising layout completeness. To address these limitations, we propose RoomCraft, a multi-stage pipeline that converts real images, sketches, or text descriptions into coherent 3D indoor scenes. Our approach combines a scene generation pipeline with a constraint-driven optimization framework. The pipeline first extracts high-level scene information from user inputs and organizes it into a structured format containing room type, furniture items, and spatial relations. It then constructs a spatial relationship network to represent furniture arrangements and generates an optimized placement sequence using a heuristic-based depth-first search (HDFS) algorithm to ensure layout coherence. To handle complex multi-constraint scenarios, we introduce a unified constraint representation that processes both formal specifications and natural language inputs, enabling flexible constraint-oriented adjustments through a comprehensive action space design. Additionally, we propose a Conflict-Aware Positioning Strategy (CAPS) that dynamically adjusts placement weights to minimize furniture collisions and ensure layout completeness. Extensive experiments demonstrate that RoomCraft significantly outperforms existing methods in generating realistic, semantically coherent, and visually appealing room layouts across diverse input modalities. Figure 1. RoomCraft generates 3D room layouts based on user-provided inputs, including real-world images, sketches, or textual descriptions. The resulting scenes exhibit high diversity and closely align with the provided inputs, featuring realistic furniture arrangements and coherent spatial layouts. 0 0 footnotetext: *Equal contribution. 0 0 footnotetext: \dagger Corresponding authors. 1. Introduction In the field of Artificial General Intelligence (AGI), explicit generation ( Fridman et al. 2023 ; Hu et al. 2024 ; Zhou et al. 2024 ; Zhang et al. 2024b ) of 3D indoor scenes is essential for accurately simulating real-world environments, providing critical benefits such as geometric consistency, physical interactivity, and editability. These explicitly modeled scenes not only enable controlled generation and support physics-based rendering (PBR) but also offer reliable environments for robotics training and Embodied AI research. However, generating plausible 3D indoor scenes remains challenging, especially in achieving realistic furniture arrangements within a space. To address these problems, researchers commit to the implicit generation methods, including diffusion-based ( Höllein et al. 2023 ; Song et al. 2023 ; Tang et al. 2024 ; Lin and Yadong 2023 ; Zhai et al. 2025 ) and LLM-based methods ( Yang et al. 2024b ; Feng et al. 2024 ; Fu et al. 2025 ; Wang et al. 2024 ; Yang et al. 2024a ) . Although these methods have shown promising results in 3D room layout generation, they may still struggle with repetitive elements and unrealistic layouts due to lacking global contextual understanding. On the other hand, explicit generation methods ( Torne et al. 2024 ; Hsu et al. 2023 ; Raistrick et al. 2024 ; Hu et al. 2024 ; Deitke et al. 2022 ) employ rule-based procedural generation to establish furniture positions and layouts. Due to the limitations of rule completeness and algorithmic constraints, these procedurally generated scenes frequently lack realism and fall short of providing the diversity needed for downstream tasks. Alternatively, Digital Cousins ( Dai et al. 2024a ) is an excellent work to extract high-level attributes from input images, such as semantic information, relative spatial relationships, and physical properties among furniture items, enabling the generation of accurate relative relationships. However, it overlooks the overall relationship between the furniture and the room structure, making it challenging to generate a complete room layout. InfineIndoor ( Raistrick et al. 2024 ) lacks visual priors and constraint information, resulting in poor controllability and visual harmony, exampling as out-of-boundary placement or inappropriate orientation. Therefore, we argue that beyond the diversity and realism , a 3D room generation method should also consider the following aspects: Controllability: The ability to generate semantically and visually coherent room scenes based on a given instruction. Completeness: The ability to generate a well-organized, complete room layout based on a specific viewpoint input. In this paper, we introduce RoomCraft, a system designed to convert user-provided real images, sketches, or text descriptions into 3D indoor scenes. RoomCraft employs a multi-stage interactive generation pipeline, beginning with the extraction of high-level scene information from user inputs and its conversion into a structured, standardized format. This information includes room type, furniture categories, and spatial relationships between objects, leveraging the vision-language models (VLMs) to process various input modalities and ensure consistency across them. RoomCraft subsequently constructs a spatial relationship network to represent the relative positions of furniture items, generating a task list to guide furniture arrangement. The task list is organized through a heuristic-based depth-first search, prioritizing the placement of furniture items with stronger spatial constraints, thus ensuring layout coherence and spatial rationality. Additionally, to address complex multi-constraint scenarios in realistic room layouts, RoomCraft incorporates a constraint-driven optimization framework with three key components: a unified constraint representation system that processes both formal specifications and natural language inputs, an intelligent action space design that handles ambiguous spatial requirements, and a Conflict-Aware Positioning Strategy (CAPS) that prevents furniture collisions through dynamic weight adjustment. Unlike existing methods that compromise layout completeness when constraints become numerous, our approach maintains both constraint satisfaction and arrangement integrity, enabling the generation of semantically coherent and spatially rational 3D indoor scenes across diverse input modalities. The contributions of this paper are summarized as follows: (1) We propose a novel 3D indoor scene generation method, RoomCraft , that creates realistic 3D indoor scenes guided by the image, sketch, or text description. (2) We introduce an adaptive layout optimization algorithm to resolve object conflicts and generate 3D indoor layouts that are spatially coherent and visually appealing. (3) We have conducted extensive experiments to generate realistic 3D rooms, which perform better than existing state-of-the-art methods. 2. Related Works Data-Driven Scene Generation. Early efforts ( Henzler et al. 2019 ; Nguyen-Phuoc et al. 2019 ; Wu et al. 2016 ; Yang et al. 2019 ; Zhou et al. 2021 ) in 3D generation focused on learning the distribution of 3D shapes and textures from category-specific datasets. These methods evolved to enable more realistic scene generation using real-world data. For instance, Scan2BIM ( Murali et al. 2017 ) uses heuristics for wall detection to generate 2D floor plans, and Ochmann ( Ochmann et al. 2019 ) applies integer linear programming for layout inference. Furukawa ( Cabral and Furukawa 2014 ) uses multi-view stereo to create 3D meshes of interiors. Recent advancements like the ACDC system ( Dai et al. 2024b ) introduce virtual environments that replicate real-world scenes’ geometric and semantic features, improving generation efficiency and transfer for diverse robotic training environments. Procedural Based Scene Generation. Researchers have explored Procedural Content Generation (PCG) for both natural and urban 3D environments ( Gasch et al. 2022 ; Zhang et al. 2019 ; Lipp et al. 2011 ; Talton et al. 2011 ; Vanegas et