SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Models
Yuxuan Zhang · 2025
Read SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Models by Yuxuan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18168.txt.
Description
Fine-tuning a large language model on new tasks can erase what it already knew, a failure called catastrophic forgetting. Yuxuan Zhang proposes SECURA, a variant of the popular low-rank adaptation method. It pairs a matrix decomposition that keeps important columns intact with a sigmoid-based scaling that protects large, meaningful weights.
Across math problems, question answering, translation, and logic tasks, SECURA improved fine-tuning scores over a strong rival method on five open models. In continual learning tests it kept more than seventy percent of the models' basic knowledge. That retention beat replay and regularization baselines. The method aims to let small teams adapt big models without retraining them or losing their skills.
Catalog record
| CREATOR | Yuxuan Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18168