Scaling Image Tokenizers with Grouped Spherical Quantization

Jiangtao Wang, Zhen Qin, Yifan Zhang, Vincent Tao Hu, Björn Ommer, Rania Briq, Stefan Kesselheim, Jülich Supercomputing Centre, TapTap, Tsinghua University, CompVis @ LMU Munich, MCML Training Code & Checkpoints · 2024

Read Scaling Image Tokenizers with Grouped Spherical Quantization by Jiangtao Wang, Zhen Qin, Yifan Zhang, Vincent Tao Hu, Björn Ommer, Rania Briq, Stefan Kesselheim, Jülich Supercomputing Centre, TapTap, Tsinghua University, CompVis @ LMU Munich, MCML Training Code & Checkpoints (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02632.txt.

Description

Jiangtao Wang and six co-authors set out Grouped Spherical Quantization in twenty twenty-four. It is a way to tokenize images for generative models. Image tokenizers squeeze pictures into discrete codes. Past designs used old adversarial training recipes. Tests of them were not even, so scaling was hard to judge. Their method starts the codebook on a sphere. It holds lookups to codes on that sphere.

Their tokenizer beat the best rivals on reconstruction quality. It did so with fewer training iterations. Scaling studies then varied latent dimension, codebook size, and compression ratio. They showed new regimes at high and low spatial compression. The method restructures high-dimensional latents into compact spaces. That lets it scale with less waste. At sixteen-fold downsampling it hit a reconstruction score the authors call state of the art.

Catalog record

CREATORJiangtao Wang, Zhen Qin, Yifan Zhang, Vincent Tao Hu, Björn Ommer, Rania Briq, Stefan Kesselheim, Jülich Supercomputing Centre, TapTap, Tsinghua University, CompVis @ LMU Munich, MCML Training Code & Checkpoints
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02632

Browse the full library · About the Tower of Babel