Continual-learning for Modelling Low-Resource Languages from Large Language Models
Santosh Srinath K, Mudit Somani, Varun Reddy Padala, Prajna Devi Upadhyay, Abhijit Das · 2026
Read Continual-learning for Modelling Low-Resource Languages from Large Language Models by Santosh Srinath K, Mudit Somani, Varun Reddy Padala, Prajna Devi Upadhyay, Abhijit Das (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05874.txt.
Description
Santosh Srinath K and 4 co-authors address in 2026 a practical obstacle in building small language models for low-resource languages from large ones: catastrophic forgetting, where new training erases older abilities. Their setting adapts large models into small multilingual models, and the forgetting problem is the central challenge they target.
Their continual learning strategy combines parts-of-speech-based code-switching with a replay adapter. Code-switching mixes languages within text, while replay revisits earlier material during training to protect it. Experiments covered vision-language tasks such as visual question answering alongside language modeling. The authors report that the proposed architecture mitigated the forgetting gap successfully in those tests, offering a route to capable small models for languages with limited data.
Catalog record
| CREATOR | Santosh Srinath K, Mudit Somani, Varun Reddy Padala, Prajna Devi Upadhyay, Abhijit Das |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05874