STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics–Physics Dual System
Zhen Luo, Yixuan Yang, Xudong Xu, Jinkun Hao, Zhaoyang Lyu, Feng Zheng, Jiangmiao Pang, Yanwei Fu · 2026
Read STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics–Physics Dual System by Zhen Luo, Yixuan Yang, Xudong Xu, Jinkun Hao, Zhaoyang Lyu, Feng Zheng, Jiangmiao Pang, Yanwei Fu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16137.txt.
Introduction
Luo and seven colleagues present STABLE, a dual system that generates simulation-ready tabletop scenes from task instructions. Language-model-only layouts often yield collisions or floating objects, since such models reason weakly in 3D space. STABLE pairs a semantic reasoner that drafts coarse layouts with a physics corrector that refines poses for physical plausibility. Alternating the two modules expands scenes from task-critical objects to background items, with far better physical validity than prior methods.
Catalog record
| CREATOR | Zhen Luo, Yixuan Yang, Xudong Xu, Jinkun Hao, Zhaoyang Lyu, Feng Zheng, Jiangmiao Pang, Yanwei Fu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58513 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16137
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics–Physics Dual System Zhen Luo, Yixuan Yang, Xudong Xu, Jinkun Hao, Zhaoyang Lyu, Feng Zheng, Jiangmiao Pang, Yanwei Fu Abstract Generating simulation-ready tabletop scenes from task instructions is an intriguing and promising research direction in the field of Embodied AI. However, existing task-to-scene generation methods rely exclusively on large language models (LLMs) to predict scene layouts, inevitably yielding object collisions or floating due to LLMs’ inherent limitations in 3D spatial reasoning. In this paper, we present STABLE , a semantics–physics dual-system tailored for simulation-ready tabletop scene generation. STABLE consists of two complementary modules: (i) a Semantic Reasoner , a fine-tuned LLM trained on a structured tabletop scene dataset to generate coarse layouts from input task instructions, and (ii) a Physics Corrector , a physics-aware flow-based denoising model that outputs pose updates to refine layouts, which ensures the physical plausibility of scenes while preserves semantic alignment with task instructions. STABLE adopts a progressive generation paradigm: by alternating between the Semantic Reasoner and Physics Corrector, it incrementally expands the scene from task-critical objects to background objects. Experiments demonstrate that STABLE successfully generates simulation-ready tabletop scenes that strictly conform to task instructions and significantly enhances the physical validity of scenes over prior art. † † affiliationnotice: *Equal contribution. †Corresponding author. STABLE.github.io 1 Introduction Figure 1 : Overview of our proposed STABLE for tabletop scene generation. STABLE is a Semantics–Physics dual system that alternates between an LLM-based Semantic Reasoner and a geometry-aware flow-based Physics Corrector to generate diverse, task-aligned, and simulation-ready tabletop layouts. Synthetic data is emerging as an increasingly vital component in both the training and evaluation phases of embodied AI, thanks to its inherent advantages of low cost and easy scalability ( 4 ; 19 ) . For synthetic data in robotic manipulation, the generation of diverse tabletop scenes that align with manipulation task instructions can provide a wide range of robotic simulation environments, thereby receiving increasing attention recently ( 8 ; 3 ) . Undoubtedly, the ability to effectively interpret high-level task instructions while ensuring the generation of physically plausible or simulation-ready scenes has become the key to this generation task. In recent years, researchers have attempted to leverage powerful large language models (LLMs) to interpret text descriptions or task instructions for the purpose of scene generation. A growing body of LLM-based scene generation methods—whether directly harnessing the zero-shot capabilities of LLMs ( 5 ) , adopting multi-step prompting strategies ( 2 ; 27 ; 17 ; 24 ) , or further fine-tuning LLMs on scene datasets ( 25 ; 26 ; 9 ) —are capable of translating high-level semantics into structured scene layouts. Despite yielding promising generation results for both room-scale and tabletop scenes, these methods still struggle to synthesize simulation-ready scenes ( 2 ; 21 ; 9 ) , often resulting in object interpenetration or floating artifacts. We attribute these failures to the hallucination phenomenon of LLMs as well as their inherent limitations in 3D spatial reasoning. Meanwhile, certain post-hoc optimization attempts can improve the physical plausibility of generated scenes to some extent ( 16 ; 28 ) . However, such optimizations inevitably impose heavy computational burdens and even fail to find a feasible solution for scenes afflicted with severe object collisions. More importantly, they may disrupt the original scene layouts, thereby resulting in cluttered or out-of-distribution layouts and potentially undermining alignment with the given task instructions ( 9 ; 16 ) . For instance, if the task instruction specifies placing an apple to the left of a banana on the tabletop, such post-hoc optimizations may erroneously displace the apple to the banana’s right. Inspired by the impressive “System 1, System 2” VLA model adopted in Helix ( 6 ) , we propose STABLE , a Semantics–Physics dual-system framework for generating task-aligned and simulation-ready tabletop scenes without compromising scene layout diversity. Specifically, STABLE integrates two complementary components: a Semantic Reasoner and a Physics Corrector . Following prior works ( 9 ) , the Semantic Reasoner is an LLM fine-tuned on MesaTask-10K via supervised fine-tuning (SFT). Given a task instruction, the Semantic Reasoner is responsible for interpreting the semantics of the instruction and generating a coarse yet task-aligned scene layout that specifies task-relevant objects and their interrelations. The Physics Corrector is a lightweight flow-based denoising model that predicts pose updates to refine the scene layout. To ensure the physical plausibility of the generated scenes, we deliberately equip the Physics Corrector with explicit geometric awareness, i.e. , conditioning it on object point clouds to predict object pose updates. Moreover, we employ subtle signed distance function (SDF) collision losses, which are highly sensitive to inter-object penetration, to effectively resolve object-object and object-table collisions. Additionally, a support-contact loss is utilized to eliminate object floating artifacts in the generated scenes. The Physics Corrector is also trained on MesaTask-10K ( 9 ) , enabling it to learn a reasonable layout distribution of tabletop scenes and thus avoid generating cluttered tabletop layouts—even when handling coarse layouts with severe collision issues. For tabletop layout synthesis, STABLE leverages a progressive inference workflow characterized by alternating iterations of the Semantic Reasoner and Physics Corrector. Specifically, the Semantic Reasoner incrementally expands the layout from task-critical objects to contextual background objects, with the Physics Corrector invoked immediately after each expansion stage to preserve the physical plausibility of the scene. This design not only mitigates error accumulation that would otherwise lead to severe object collisions but also inherently enables incremental scene editing and arrangement. We summarize our main contributions as follows: (1) We propose STABLE, a first-of-its-kind dual-system framework for scene generation, which directly maps task instructions to simulation-ready tabletop layouts by decoupling semantic layout generation from physics-aware pose correction. (2) We introduce a geometry-aware denoising target, which allows the Physics Corrector to handle complex or challenging spatial relationships such as stacking and containment. (3) Extensive experiments demonstrate that STABLE can generate simulation-ready tabletop scenes with strict alignment to task instructions, achieving substantial improvements in both the physical validity and task alignment of scenes compared to existing methods. 2 Related Work 2.1 3D Layout Generation Recent advances in 3D scene layout generation can be broadly grouped into (i) prompting-driven approaches built on proprietary foundation models and (ii) data-driven approaches trained on curated 3D layout datasets. Prompting-driven methods ( 5 ; 14 ; 2 ; 27 ; 17 ; 24 ) typically rely on large closed-source LLMs and carefully designed prompting rules or multi-round interactions to elicit spatial reasoning and produce structured layouts. While these methods demonstrate strong semantic priors, they largely operate at the textual level and provide limited mechanisms for obtaining geometry-aware feedback from the generated layouts, making it difficult to consistently enforce physical plausibility. A related line of work ( 21 ; 20 ; 10 ) , introduces intermediate image representations to bridge text and 3D scenes, using vision signals to help models interpret coordinates and spatial arrangements. However, such long-horizon pipelines often suffer from error accumulation and increased cost, and can be especially brittle under cluttered tabletop scenes with occlusions and stacking. On the other hand, the availability of large-scale 3D datasets such as 3D-FRONT ( 7 ) and MesaTask ( 9 ) has enabled data-driven learning of layout distributions. Methods like