LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation
Xin Li, Anand D. Sarwate · 2025
Read LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation by Xin Li, Anand D. Sarwate (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13568.txt.
Description
Tuning a huge trained model for a new task can be done by training a small patch instead of the whole net. Xin Li and Anand Sarwate shrink that patch far more in 2025. Their trick borrows an old idea from number work: some matrices can be split into a few simple layers, a form with low separation rank.
Common patches already use a low-rank split to cut their size. As models grow, even that patch grows large. The new kernel wraps the patch's matrices in the layered form, which cuts the count of trainable values to about half of the past best. Tests report state-of-the-art scores at that smaller size. The gain is plain thrift: less memory, less time, and a patch that is cheap to store per task.
Catalog record
| CREATOR | Xin Li, Anand D. Sarwate |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13568