Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration
Farhad Pourkamali-Anaraki · 2026
Read Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration by Farhad Pourkamali-Anaraki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02659.txt.
Description
Pourkamali-Anaraki improves low-rank compression of pretrained models. Exact singular value decomposition costs too much for large weight matrices. Randomized SVD falters when singular values decay slowly, a common case in modern models. His randomized subspace iteration adds power steps to sharpen spectral separation and control error. He also ties compression error to prediction error through softmax perturbations, and tests the method on convolutional networks. The goal is smaller models that still answer well.
Catalog record
| CREATOR | Farhad Pourkamali-Anaraki |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34933 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.02659