Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks
Naimul Haque · 2025
Read Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks by Naimul Haque (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01241.txt.
Description
Naimul Haque studies how language models forget old skills in a 2025 computing paper. When a model learns tasks one after another, new training can erase old gains. That failure, called catastrophic forgetting, blocks models that must keep adapting. Haque fine-tunes several open models, all under ten billion parameters, on a chain of language tasks.
The tasks come from a standard bench and cover sentiment, paraphrase, grammar, and inference. Prompt tricks and task-specific tweaks were used during training. One small model forgot very little while still learning well. Two others also kept a strong balance after tuning. The results give builders a short list of models suited to steady lifelong learning.
Catalog record
| CREATOR | Naimul Haque |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 14 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-01241