A More Word-like Image Tokenization for MLLMs

Hyun Lee, Hyemin Jeong, Yejin Kim, Hyungwook Choi, Hyunsoo Cho, Soo Kyung Kim, Joonseok Lee · 2026

Read A More Word-like Image Tokenization for MLLMs by Hyun Lee, Hyemin Jeong, Yejin Kim, Hyungwook Choi, Hyunsoo Cho, Soo Kyung Kim, Joonseok Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17954.txt.

Description

Hyun Lee and six colleagues propose Disentangled Visual Tokenization, a more word-like way to tokenize images for multimodal models. Projectors map pixels into streams of continuous, correlated embeddings, unlike the discrete tokens language models expect. DiVT clusters patch embeddings into coherent semantic units, so each token matches a visual concept rather than a grid cell. It adapts its token budget to image complexity, without changing encoder or language model. Across benchmarks, DiVT matches or beats baselines with far fewer tokens, cutting memory cost and latency.

Catalog record

CREATORHyun Lee, Hyemin Jeong, Yejin Kim, Hyungwook Choi, Hyunsoo Cho, Soo Kyung Kim, Joonseok Lee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53287
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17954

Browse the full library · About the Tower of Babel