Decorum : A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes
Kelly O. Marshall, Omid Poursaeed, Sergiu Oprea, Amit Kumar, Anushrut Jignasu, Chinmay Hegde, Yilei Li, Rakesh Ranjan · 2025
Read Decorum : A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes by Kelly O. Marshall, Omid Poursaeed, Sergiu Oprea, Amit Kumar, Anushrut Jignasu, Chinmay Hegde, Yilei Li, Rakesh Ranjan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18155.txt.
Introduction
Kelly Marshall and co-authors present a language-based method to design indoor 3D scenes in a chosen style. Their 2025 system, called Decorum, lets users guide layout with natural language. Past tools took only simple object lists or pairwise place relations, which gave users little style control.
Decorum keeps text forms at each stage, so large language models can map prompts to scene plans. Furniture is picked by a new retrieval step that uses multimodal models to match items to the brief. Tests on a standard scene data set show gains over past work in both scene quality and item choice. The authors argue that text at every stage gives designers finer, more flexible control.
Catalog record
| CREATOR | Kelly O. Marshall, Omid Poursaeed, Sergiu Oprea, Amit Kumar, Anushrut Jignasu, Chinmay Hegde, Yilei Li, Rakesh Ranjan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18155
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Decorum : A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes Kelly O. Marshall, Omid Poursaeed, Sergiu Oprea, Amit Kumar, Anushrut Jignasu, Chinmay Hegde, Yilei Li, Rakesh Ranjan Abstract 3D indoor scene generation is an important problem for the design of digital and real-world environments. To automate this process, a scene generation model should be able to not only generate plausible scene layouts, but also take into consideration visual features and style preferences. Existing methods for this task exhibit very limited control over these attributes, only allowing text inputs in the form of simple object-level descriptions or pairwise spatial relationships. Our proposed method Decorum enables users to control the scene generation process with natural language by adopting language-based representations at each stage. This enables us to harness recent advancements in Large Language Models (LLMs) to model language-to-language mappings. In addition, we show that using a text-based representation allows us to select furniture for our scenes using a novel object retrieval method based on multimodal LLMs. Evaluations on the benchmark 3D-FRONT dataset show that our methods achieve improvements over existing work in text-conditioned scene synthesis and object retrieval. † † affiliation: Meta Reality Labs † † affiliation: New York University † † affiliation: Iowa State University † † contribution: Work done at Meta † † contribution: Joint last author † † date: August 9, 2026 † † correspondence: Kelly Marshall at [email protected] † † Code: Code will be released upon acceptance Figure 1 : Illustration of the Decorum pipeline. We finetune LLaMA to obtain both Prompt2Annotation and Annotation2Layout models to convert from user prompts to CSS scene layouts in a two-stage process. Using this intermediate annotation representation decouples the generation of spatial and stylistic elements, allowing us to separately perform furniture selection using the tagged objects. Purple coloring indicates language model modules that we train using LoRA and green coloring shows our DecoRate furniture retrieval method which relies entirely on pretrained LLMs. 1 Introduction The generation of 3D indoor scenes is a problem which has garnered considerable attention, with potential applications in video game development, automated interior design, and metaverse content creation ( Merrell et al. 2011 ; Fisher et al. 2015 ; Qi et al. 2018 ; Wang et al. 2018 ; Ritchie et al. 2019 ; Zhang et al. 2020 ; Yang et al. 2021b ; Yang et al. 2021a ; Höllein et al. 2023 ; Song et al. 2023 ; Cohen-Bar et al. 2023 ; Lin et al. 2023 ; Feng et al. 2023 ; Patil et al. 2023 ) . In these applications, it is critical that the generation process produces scenes that are practical as well as visually pleasing. While existing works Liu et al. 2023b ; Lin and Mu 2024 have taken into account visual coherence, this does not equate to producing visually pleasing scenes. This is because there does not exist a single objective standard for what constitutes an aesthetically appealing scene. Instead, it is a matter of individual preference. As a result, visual quality cannot be fully addressed by unconditional scene generation methods which do not incorporate user input. To better integrate user input, the scene generation process can be conditioned on natural language. This is a flexible and powerful interface which can allow users to describe their desired scenes in as much or as little detail as they desire. A description may include both style and spatial details, such as object and scene styles, as well as practical layout information. Existing text-conditioned methods lack the generality to handle both the style and structure of a scene in a text prompt, restricting user specifications to either brief object-level descriptions (e.g. color) Liu et al. 2023b or explicit pairwise spatial relationships Lin and Mu 2024 . To fill this gap, we propose Decorum , a generative framework for synthesizing indoor 3D scenes from user prompts. Decorum decouples the generation of spatial and stylistic elements by first converting the user prompt into a dense annotation with object tags. Using this detailed annotation, Decorum separately designs a layout for the scene’s objects and assigns each tagged object a textured mesh retrieved from an inventory of existing furniture. We provide an overview of our proposed model pipeline in Figure 1 . Our method uses language-based representations for 3D scenes at each stage, allowing us to fully harness the power of Large Language Models (LLMs) towards scene generation. The scenes we output are in CSS format, following previous work Feng et al. 2023 which shows that this structured representation facilitates spatial understanding in LLMs. Both the mapping from prompt to annotation and annotation to layout are learned using low rank adaptations Hu et al. 2021 of the open-source LLaMA 3.1 Touvron et al. 2023 Our use of language representations carries the additional benefit of allowing text-based search for furniture retrieval. Existing methods either generate only object category and size Paschalidou et al. 2021 ; Feng et al. 2023 or search over an inventory of textured meshes by using predicted object embeddings Lin and Mu 2024 ; Liu et al. 2023b . Using the object tags in our generated scene annotation, we obtain detailed descriptions of each object in the scene, combining any descriptions of it included in the prompt with the information provided about the scene as a whole. This gives an informative object representation for performing text-based object retrieval. To make full use of this representation, we look at recent developments in Large Multimodal Models (LMMs) which ground LLMs’ reasoning and world knowledge to visual information. Leveraging this link from textual to visual features, we devise a new scoring method to compute an object-text similarity metric which can be used for object-retrieval. We dub this retrieval method DecoRate, and show that its sensitivity to fine-grained image details leads to major improvements over CLIP-based object retrieval. To alleviate the computational burden of rating a potentially expansive inventory of objects with LMM inferences, we show the effectiveness of a coarse-to-fine pipeline which relies on CLIP to generate a small number of search candidates. Integrating DecoRate into our Decorum pipeline results in a scene generation method which combines the spatial reasoning, natural language understanding, and multimodal capabilities of LLMs to comprehend and design scenes in line with varied user inputs. We evaluate Decorum on the benchmark 3D-FRONT dataset Fu et al. 2020a , which consists of professionally arranged indoor scenes. Through quantitative and qualitative comparisons with existing works we show that our method has both the flexibility to represent a diverse range of scenes and the controllability to reflect a wide range of user inputs. We also separately verify the effectiveness of our DecoRate method for furniture retrieval. Paper Contributions. The contributions of our work are summarized as follows: • We propose Decorum , a method for synthesizing indoor 3D scenes from user descriptions by leveraging a semi-structured intermediate representation. • We introduce DecoRate, a simple yet powerful text-based search method for object ranking and retrieval which achieves a 15x improvement in Top-1 retrieval accuracy compared to existing vector-similarity based methods. • We demonstrate that our pipeline for text-conditioned scene generation outperforms existing methods on the benchmark 3D-FRONT dataset. In addition, we introduce the Text Fidelity Ranking (TFR) metric