The Effects of Grouped Structural Global Pruning of Vision Transformers on Domain Generalisation
Hamza Riaz and Alan F. Smeaton · 2025
Read The Effects of Grouped Structural Global Pruning of Vision Transformers on Domain Generalisation by Hamza Riaz and Alan F. Smeaton (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04196.txt.
Description
Hamza Riaz and Alan Smeaton study in 2025 what happens when most of a vision transformer is cut away before it is used on new kinds of images. These models are large, which makes them hard to run on small devices. Their cut method removes whole groups of parts that a dependency analysis marks as spare.
They apply the cuts at three depths: half, three quarters, and nearly all of the spare parts. The cut models are then tuned on some image sets and tested on others they have not seen, across two standard test suites. The question is whether a lean model still copes when the image style shifts, as it does between photos, art, and sketches. A cut that keeps that skill would let strong vision models run on phones and cameras.
Catalog record
| CREATOR | Hamza Riaz and Alan F. Smeaton |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-04196