Continual Fine-Tuning of Large Language Models via Program Memory
Hung Le, Svetha Venkatesh · 2026
Read Continual Fine-Tuning of Large Language Models via Program Memory by Hung Le, Svetha Venkatesh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13162.txt.
Description
Le and Venkatesh propose ProCL, a LoRA setup with program memory inspired by brain learning. Standard LoRA updates struggle to balance adaptation with retention when models update in sequence on small data. ProCL organises adapters into memory slots, fetched through input-based attention, so similar inputs reuse shared regions while unused capacity waits for data. Slots combine with the base adapter, which slowly gathers knowledge across tasks. The method runs within LoRA at no extra cost, improving retention and cutting catastrophic forgetting versus other continual LoRA plans.
Catalog record
| CREATOR | Hung Le, Svetha Venkatesh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51647 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13162