Planning with Unified Multimodal Models
Yihao Sun, Zhilong Zhang, Yang Yu, Pierre-Luc Bacon · 2025
Read Planning with Unified Multimodal Models by Yihao Sun, Zhilong Zhang, Yang Yu, Pierre-Luc Bacon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23014.txt.
Description
Yihao Sun and 3 co-authors plan robot actions with unified multimodal models that both take in and produce images and text. Most prior planners reason in words alone, which limits what they can foresee. Their framework, Uni-Plan, lets one model play three roles at once. It acts as the policy, the world dynamics model, and the value judge.
Imagined futures can contain fake dynamics, so they add a guard. The same model grades its own imagined steps and filters out invalid ones. On embodied tasks, Uni-Plan raised success rates well past vision-language baselines. It needed no expert demos and scaled well with more training data.
Catalog record
| CREATOR | Yihao Sun, Zhilong Zhang, Yang Yu, Pierre-Luc Bacon |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23014