Mitigating Premature Discretization with Progressive Quantization for Robust Vector Tokenization
Wenhao Zhao, Qiran Zou, Zhouhan Lin, Dianbo Liu · 2026
Read Mitigating Premature Discretization with Progressive Quantization for Robust Vector Tokenization by Wenhao Zhao, Qiran Zou, Zhouhan Lin, Dianbo Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22304.txt.
Description
Zhao and co-authors fix a timing flaw in vector quantization, a core step in multimodal models. Standard methods force data into discrete codes before the encoder has learned the data's shape. The authors call this premature discretization. Their method, ProVQ, treats quantization as a curriculum. Training slides smoothly from a continuous latent space to a discrete one. Tests span images and protein sequences. On ImageNet sets, ProVQ improves reconstruction and generation scores. On protein structure tokens, it sets a new best mark.
Catalog record
| CREATOR | Wenhao Zhao, Qiran Zou, Zhouhan Lin, Dianbo Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34900 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.22304