Uni-LoRA: One Vector is All You Need
Kaiyang Li, Shaobo Han, Qing Su, Wei Li, Zhipeng Cai, Shihao Ji · 2025
Read Uni-LoRA: One Vector is All You Need by Kaiyang Li, Shaobo Han, Qing Su, Wei Li, Zhipeng Cai, Shihao Ji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00799.txt.
Description
Kaiyang Li and co-authors unify a family of cheap fine-tuning methods in a 2025 paper. LoRA adapts large language models by training only small low-rank updates, keeping costs down. Variants push further by tying or sharing parameters, each with its own recipe. The authors show all of them fit one frame: the full update can be rebuilt from a projection out of a much smaller space.
What differs between methods is only the projection used. Most project layer by layer, which blocks sharing across the whole model. Their projection preserves lengths, allowing one shared space for every layer. The result is startling: a single trainable vector can rebuild the update for an entire model. Across language benchmarks, the one-vector method matched or beat its rivals while using the fewest parameters.
Catalog record
| CREATOR | Kaiyang Li, Shaobo Han, Qing Su, Wei Li, Zhipeng Cai, Shihao Ji |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00799