Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang · 2026
Read Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters by Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00428.txt.
Description
Wang and co-authors test if fine budget steps change how cheap model add-ons trade cost for skill. Rank sets the step size for most low-rank add-ons in use. One rank more can add thousands of train weights at a stroke. That leaves wide gaps in what low budgets can pick.
Their test add-ons split weights in a block form with small steps. One such block step stores near 21 times fewer weights than one rank step. Tests pit these add-ons with LoRA on a set model size. Three task sets and matched rules keep the race fair. The fine-step tool trains in a firm way and fills the rank gaps. Gains shift by task, so fine steps aid checks more than wins.
Catalog record
| CREATOR | Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00428