Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks

Naimul Haque · 2025

Read Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks by Naimul Haque (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01241.txt.

Description

Naimul Haque studies how language models forget old skills in a 2025 computing paper. When a model learns tasks one after another, new training can erase old gains. That failure, called catastrophic forgetting, blocks models that must keep adapting. Haque fine-tunes several open models, all under ten billion parameters, on a chain of language tasks.

The tasks come from a standard bench and cover sentiment, paraphrase, grammar, and inference. Prompt tricks and task-specific tweaks were used during training. One small model forgot very little while still learning well. Two others also kept a strong balance after tuning. The results give builders a short list of models suited to steady lifelong learning.

Catalog record

CREATORNaimul Haque
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01241

Browse the full library · About the Tower of Babel