G uided E nd-to-End A uto R egression for Image Synthesis
Bin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,† · 2026
Read G uided E nd-to-End A uto R egression for Image Synthesis by Bin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-32039.txt.
Description
Lin and ten colleagues train the two halves of an autoregressive image generator together instead of in sequence. The usual recipe trains a vector-quantized tokenizer for reconstruction, freezes it, then trains the generator on its codes. The frozen tokenizer never learns which code patterns the generator finds easy to predict. Joint training is blocked because the discrete code fed to the generator is non-differentiable, and the standard straight-through estimator collapses.
GEAR splits the codebook read-out into two branches to get around that blockage. A hard one-hot branch trains the generator by next-token prediction, while a differentiable soft branch carries a representation-alignment loss back to the tokenizer alone. On ImageNet, convergence of the generation quality metric speeds up by as much as tenfold against a strong baseline.
Catalog record
| CREATOR | Bin Lin 1 , 2 , ∗ {}^{1,2,^{*}} Zheyuan Liu 1 , 2 , ∗ {}^{1,2,^{*}} Chenguo Lin 1 Sixiang Chen 2 , ∗ {}^{2,^{*}} Yunyang Ge 1 , 2 , ∗ {}^{1,2,^{*}} Yunlong Lin Jianwei Zhang 2 Miles Yang 2 Zhao Zhong 2 Liefeng Bo 2 Li Yuan 1,† |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.32039