CORP : C losed-Form O ne-shot R epresentation- P reserving Structured Pruning for Vision Transformers
Boxiang Zhang, Baijian Yang · 2026
Read CORP : C losed-Form O ne-shot R epresentation- P reserving Structured Pruning for Vision Transformers by Boxiang Zhang, Baijian Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05243.txt.
Description
Pruning a vision transformer usually means retraining it, which defeats the purpose when a model must be slimmed quickly after training. Boxiang Zhang and Baijian Yang present CORP in this 2026 machine learning paper, a one-shot structured pruning method that needs no gradients, no fine-tuning, and only unlabeled calibration images.
The insight is to treat pruning as representation recovery. Components marked for removal are modeled as affine functions of the components that stay, and closed-form ridge regression solutions fold the compensation directly into the remaining weights. On ImageNet with the DeiT family, experiments exposed heavy redundancy in both the feed-forward and attention parts. A huge DeiT model kept accuracy above 83 percent after half of both structures were pruned away. The closed-form recipe makes post-training compression a calculation rather than a project.
Catalog record
| CREATOR | Boxiang Zhang, Baijian Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-05243