CORP : C losed-Form O ne-shot R epresentation- P reserving Structured Pruning for Vision Transformers

Boxiang Zhang, Baijian Yang · 2026

Read CORP : C losed-Form O ne-shot R epresentation- P reserving Structured Pruning for Vision Transformers by Boxiang Zhang, Baijian Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05243.txt.

Description

Pruning a vision transformer usually means retraining it, which defeats the purpose when a model must be slimmed quickly after training. Boxiang Zhang and Baijian Yang present CORP in this 2026 machine learning paper, a one-shot structured pruning method that needs no gradients, no fine-tuning, and only unlabeled calibration images.

The insight is to treat pruning as representation recovery. Components marked for removal are modeled as affine functions of the components that stay, and closed-form ridge regression solutions fold the compensation directly into the remaining weights. On ImageNet with the DeiT family, experiments exposed heavy redundancy in both the feed-forward and attention parts. A huge DeiT model kept accuracy above 83 percent after half of both structures were pruned away. The closed-form recipe makes post-training compression a calculation rather than a project.

Catalog record

CREATORBoxiang Zhang, Baijian Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05243

Browse the full library · About the Tower of Babel