G uided E nd-to-End A uto R egression for Image Synthesis

Bin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,† · 2026

Read G uided E nd-to-End A uto R egression for Image Synthesis by Bin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-32039.txt.

Description

Lin and ten colleagues train the two halves of an autoregressive image generator together instead of in sequence. The usual recipe trains a vector-quantized tokenizer for reconstruction, freezes it, then trains the generator on its codes. The frozen tokenizer never learns which code patterns the generator finds easy to predict. Joint training is blocked because the discrete code fed to the generator is non-differentiable, and the standard straight-through estimator collapses.

GEAR splits the codebook read-out into two branches to get around that blockage. A hard one-hot branch trains the generator by next-token prediction, while a differentiable soft branch carries a representation-alignment loss back to the tokenizer alone. On ImageNet, convergence of the generation quality metric speeds up by as much as tenfold against a strong baseline.

Catalog record

CREATORBin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,†
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.32039

Browse the full library · About the Tower of Babel