Context Parametrization with Compositional Adapters
Josip Jukić, Martin Tutek, Jan Šnajder · 2025
Read Context Parametrization with Compositional Adapters by Josip Jukić, Martin Tutek, Jan Šnajder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22158.txt.
Description
Josip Jukić, Martin Tutek, and Jan Šnajder present CompAs, a way to turn context for a language model into small plug-in modules called adapters. Models normally adapt to new tasks by reading examples in the prompt, which wastes space, or by fine-tuning, which costs training and loses flexibility. Earlier work generated adapters from a single chunk of context. Real tasks, though, often need several chunks of information merged together.
CompAs is a meta-learning framework whose generated adapters have a compositional structure: they can be merged by simple algebra. That yields three gains. Inference gets cheaper, behavior stays steadier under long contexts, and inputs longer than the model's window gain a principled handling. On multiple-choice and extractive question answering tasks, CompAs beat in-context learning and earlier generator-based methods, with the lead growing as inputs multiplied.
Catalog record
| CREATOR | Josip Jukić, Martin Tutek, Jan Šnajder |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22158