Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation

Junwei Zhou, Yu-Wing Tai · 2026

Read Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation by Junwei Zhou, Yu-Wing Tai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25326.txt.

Description

Junwei Zhou and Yu-Wing Tai turn single-image 3D layout estimation into a perceive-then-plan task. A Perceiver first grounds objects in 3D, then a Planner refines the scene for physical plausibility while staying faithful to the image. Their Layout-as-Policy approach treats layouts as states and corrections as actions like shifting, rotating, and rescaling. Training mixes trajectory starts with preference tuning, so the model learns fixes without hand-built rewards. Experiments show layouts that hold together physically and match observations, with support for editing and manipulation.

Catalog record

CREATORJunwei Zhou, Yu-Wing Tai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54644
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25326

Browse the full library · About the Tower of Babel