The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling

Takuya Shiba · 2026

Read The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling by Takuya Shiba (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03191.txt.

Description

Shiba explains why bigger vision encoders stop helping robot action models once actions are squeezed into discrete tokens. He names the effect the Compression Gap: performance follows the tightest bottleneck, so a fixed codebook caps gains no matter how rich the visual input becomes. Experiments on the LIBERO benchmark back the idea, since widening the codebook partly restores encoder sensitivity.

Catalog record

CREATORTakuya Shiba
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33339
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03191

Browse the full library · About the Tower of Babel