Beyond Low-Rank Tuning: Model Prior-Guided Rank Allocation for Effective Transfer in Low-Data and Large-Gap Regimes.
Chuyan Zhang, Kefan Wang, Yun Gu* · 2025
Read Beyond Low-Rank Tuning: Model Prior-Guided Rank Allocation for Effective Transfer in Low-Data and Large-Gap Regimes. by Chuyan Zhang, Kefan Wang, Yun Gu* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00327.txt.
Description
Chuyan Zhang, Kefan Wang, and Yun Gu improve a cheap way to adapt large models in a 2025 paper. The cheap method tweaks only small low-rank parts, which saves cost but can fail when the new field is far from the old one. Past fixes search for ranks at high cost or add extra rules.
Their method reads a rank clue straight from the trained weights of each layer. Layers whose weights hold more depth get more rank, and layers with less get less, with no search at all. Tests on few-shot tasks with wide field gaps beat recent rival methods on the trade-off of skill and cost.
Catalog record
| CREATOR | Chuyan Zhang, Kefan Wang, Yun Gu* |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-00327