PnP-U3D : Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generation
Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan · 2026
Read PnP-U3D : Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generation by Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03533.txt.
Description
Models that both understand and generate content are routine in 2D, but 3D has resisted unification. Yongwei Chen and 6 co-authors present PnP-U3D in a 2026 paper, a plug-and-play framework joining 3D understanding with 3D generation. Forcing everything through one autoregressive model, they argue, degrades quality and costs too much training.
Instead, understanding uses autoregressive next-token prediction, while generation uses a continuous diffusion model. A lightweight transformer bridges the language model's features and the diffusion model's conditions, letting the two exchange information without erasing their pretrained strengths. Across benchmarks the framework reaches state-of-the-art results in 3D understanding and generation, and it also performs strongly on 3D editing tasks.
Catalog record
| CREATOR | Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03533