Continual Fine-Tuning of Large Language Models via Program Memory

Hung Le, Svetha Venkatesh · 2026

Read Continual Fine-Tuning of Large Language Models via Program Memory by Hung Le, Svetha Venkatesh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13162.txt.

Description

Le and Venkatesh propose ProCL, a LoRA setup with program memory inspired by brain learning. Standard LoRA updates struggle to balance adaptation with retention when models update in sequence on small data. ProCL organises adapters into memory slots, fetched through input-based attention, so similar inputs reuse shared regions while unused capacity waits for data. Slots combine with the base adapter, which slowly gathers knowledge across tasks. The method runs within LoRA at no extra cost, improving retention and cutting catastrophic forgetting versus other continual LoRA plans.

Catalog record

CREATORHung Le, Svetha Venkatesh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51647
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13162

Browse the full library · About the Tower of Babel