PnP-U3D : Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generation

Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan · 2026

Read PnP-U3D : Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generation by Yongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03533.txt.

Description

Models that both understand and generate content are routine in 2D, but 3D has resisted unification. Yongwei Chen and 6 co-authors present PnP-U3D in a 2026 paper, a plug-and-play framework joining 3D understanding with 3D generation. Forcing everything through one autoregressive model, they argue, degrades quality and costs too much training.

Instead, understanding uses autoregressive next-token prediction, while generation uses a continuous diffusion model. A lightweight transformer bridges the language model's features and the diffusion model's conditions, letting the two exchange information without erasing their pretrained strengths. Across benchmarks the framework reaches state-of-the-art results in 3D understanding and generation, and it also performs strongly on 3D editing tasks.

Catalog record

CREATORYongwei Chen, Tianyi Wei, Yushi Lan, Zhaoyang Lyu, Shangchen Zhou, Xudong Xu, Xingang Pan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03533

Browse the full library · About the Tower of Babel