Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters

Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang · 2026

Read Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters by Xinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00428.txt.

Description

Wang and co-authors test if fine budget steps change how cheap model add-ons trade cost for skill. Rank sets the step size for most low-rank add-ons in use. One rank more can add thousands of train weights at a stroke. That leaves wide gaps in what low budgets can pick.

Their test add-ons split weights in a block form with small steps. One such block step stores near 21 times fewer weights than one rank step. Tests pit these add-ons with LoRA on a set model size. Three task sets and matched rules keep the race fair. The fine-step tool trains in a firm way and fills the rank gaps. Gains shift by task, so fine steps aid checks more than wins.

Catalog record

CREATORXinjue Wang, Xiuheng Wang, Yejun Zhang, Sergiy A. Vorobyov, Esa Ollila, Zhi-Yong Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00428

Browse the full library · About the Tower of Babel