Compress image to patches for Vision Transformer

Xinfeng Zhao, Yaoru Sun · 2025

Read Compress image to patches for Vision Transformer by Xinfeng Zhao, Yaoru Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10120.txt.

Description

Xinfeng Zhao and Yaoru Sun redesign how a vision transformer, in a 2025 preprint, cuts an image into patches. Vision transformers split pictures into tiles and reason over the tile sequence. Deeper models and bigger images make that cost explode. The tiling step is blind too: it ignores what the image shows. Their module, CI2P, compresses the image first and builds the tile sequence from the compressed form.

The new module drops into models in place of the old patch step. It cuts the number of tiles fed to the attention layers to a quarter. On a dataset of animal images, a model trained from scratch with CI2P beat the standard baseline by over three percent in accuracy. It also used about sixty-three percent fewer operations, and training ran twice as fast on the same hardware. The authors credit the compression stage's sense of local structure.

Catalog record

CREATORXinfeng Zhao, Yaoru Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10120

Browse the full library · About the Tower of Babel