Skill Neologisms: Towards Skill-based Continual Learning
Antonin Berthon, Nicolas Astorga, Mihaela van der Schaar · 2026
Read Skill Neologisms: Towards Skill-based Continual Learning by Antonin Berthon, Nicolas Astorga, Mihaela van der Schaar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04970.txt.
Description
Antonin Berthon, Nicolas Astorga, and Mihaela van der Schaar explore teaching language models new skills through special soft tokens they call skill neologisms. Adding skills by fine-tuning risks erasing old ones, while prompt-based methods are limited by context length and expressiveness. A learned token, by contrast, can be added to the vocabulary without changing model weights.
The team first shows pretrained models already carry tokens tied to procedural knowledge. On a controlled synthetic task, learned skill tokens improved the target skill and combined with unfamiliar skills, and separately trained tokens composed without further training. A natural-language benchmark confirmed the zero-shot composition, suggesting a scalable path to continual learning.
Catalog record
| CREATOR | Antonin Berthon, Nicolas Astorga, Mihaela van der Schaar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04970