Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration

Farhad Pourkamali-Anaraki · 2026

Read Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration by Farhad Pourkamali-Anaraki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02659.txt.

Description

Pourkamali-Anaraki improves low-rank compression of pretrained models. Exact singular value decomposition costs too much for large weight matrices. Randomized SVD falters when singular values decay slowly, a common case in modern models. His randomized subspace iteration adds power steps to sharpen spectral separation and control error. He also ties compression error to prediction error through softmax perturbations, and tests the method on convolutional networks. The goal is smaller models that still answer well.

Catalog record

CREATORFarhad Pourkamali-Anaraki
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34933
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.02659

Browse the full library · About the Tower of Babel