Post-Optimization Adaptive Rank Allocation for LoRA

Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. Srijith · 2026

Read Post-Optimization Adaptive Rank Allocation for LoRA by Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. Srijith (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27796.txt.

Description

Kumaravelu, Gupta, and Srijith propose Post-Optimization Adaptive Rank Allocation, a data-free Low-Rank Adaptation trimming method fitting fine-tuning pipelines. Standard LoRA sets the same rank across layers, ignoring inner depth and causing waste. PARA prunes LoRA ranks using a global core-value threshold across layers, yielding non-uniform ranks based on layer-wise signal weight. As a post-hoc method, PARA avoids training changes and wobble, cutting weights by 75 to 90 percent while keeping scores across vision and language benchmarks.

Catalog record

CREATORVishnuprasadh Kumaravelu, Sunil Gupta, P. K. Srijith
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42774
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.27796

Browse the full library · About the Tower of Babel