The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling
Takuya Shiba · 2026
Read The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling by Takuya Shiba (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03191.txt.
Description
Shiba explains why bigger vision encoders stop helping robot action models once actions are squeezed into discrete tokens. He names the effect the Compression Gap: performance follows the tightest bottleneck, so a fixed codebook caps gains no matter how rich the visual input becomes. Experiments on the LIBERO benchmark back the idea, since widening the codebook partly restores encoder sensitivity.
Catalog record
| CREATOR | Takuya Shiba |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33339 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03191