Unlocking [CLS] Features for Continual Post-Training
Murat Onur Yildirim, Elif Ceren Gok Yildirim, Joaquin Vanschoren · 2025
Read Unlocking [CLS] Features for Continual Post-Training by Murat Onur Yildirim, Elif Ceren Gok Yildirim, Joaquin Vanschoren (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14762.txt.
Description
AI models that learn task after task tend to forget the early ones, a tension called the stability-plasticity trade-off. Murat Onur Yildirim and co-authors propose a light fix in this 2025 paper. Big foundation models already perform well, so the fix should change as little as possible. Their module learns new task knowledge through a small adapter paired with a calibrator.
For each new task, a sparse copy of that module sits on the model's summary token, the token that gathers a whole input's meaning. The rest of the model, and its broad skills, stay untouched. The authors call the setup TOSCA. Sparse, token-level tuning aims to add knowledge without erasing the old. Such methods matter for models that must keep growing in the field, from medical scans to new product classes.
Catalog record
| CREATOR | Murat Onur Yildirim, Elif Ceren Gok Yildirim, Joaquin Vanschoren |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14762