FlexTok : Resampling Images into 1D Token Sequences of Flexible Length

Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oğuzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, Afshin Dehghan · 2025

Read FlexTok : Resampling Images into 1D Token Sequences of Flexible Length by Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oğuzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, Afshin Dehghan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13967.txt.

Description

Roman Bachmann and eight co-authors published this paper in 2025. It introduces FlexTok, a way to compress images into token sequences whose length adapts to the picture. Image generators work on tokens, and fixed-length tokenizers waste effort on simple images while starving complex ones.

FlexTok squeezes an image into anywhere from one to two hundred fifty-six ordered tokens, arranged coarse to fine like a summary that keeps adding detail. A flow-based decoder rebuilds a plausible image from any prefix of the sequence, trained by randomly dropping token tails. In generation tests with a transformer, quality scores stayed strong from eight to one hundred twenty-eight tokens, beating a fixed tokenizer with fewer tokens. Text-conditioned generation worked too. The authors note the tokens behave like a visual vocabulary, describing scenes from gist to detail. Adaptive length, they argue, should be the default for image tokenization.

Catalog record

CREATORRoman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oğuzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, Afshin Dehghan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13967

Browse the full library · About the Tower of Babel