UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models
Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao , Yining Wang · 2025
Read UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models by Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao , Yining Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20154.txt.
Description
Xueyan Zhang and co-authors fine tune large language models by selectively resetting rows and columns of frozen weight matrices, guided by vector size, in work from twenty twenty-five. Full fine tuning is costly, so lean adapters train a small slice instead. Popular low rank methods still carry more trainable weights than needed.
Their method, UORA, reparametrizes the frozen matrices through interpolation and refreshes chosen lines in an even, orthogonal fashion. The trainable count drops well below the leading low rank baseline, with better compute and storage economy than a shared vector rival. Benchmarks on language understanding, generation, instruction tuning, and image classification held accuracy with negligible overhead.
Catalog record
| CREATOR | Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao , Yining Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20154