Context Parametrization with Compositional Adapters

Josip Jukić, Martin Tutek, Jan Šnajder · 2025

Read Context Parametrization with Compositional Adapters by Josip Jukić, Martin Tutek, Jan Šnajder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22158.txt.

Description

Josip Jukić, Martin Tutek, and Jan Šnajder present CompAs, a way to turn context for a language model into small plug-in modules called adapters. Models normally adapt to new tasks by reading examples in the prompt, which wastes space, or by fine-tuning, which costs training and loses flexibility. Earlier work generated adapters from a single chunk of context. Real tasks, though, often need several chunks of information merged together.

CompAs is a meta-learning framework whose generated adapters have a compositional structure: they can be merged by simple algebra. That yields three gains. Inference gets cheaper, behavior stays steadier under long contexts, and inputs longer than the model's window gain a principled handling. On multiple-choice and extractive question answering tasks, CompAs beat in-context learning and earlier generator-based methods, with the lead growing as inputs multiplied.

Catalog record

CREATORJosip Jukić, Martin Tutek, Jan Šnajder
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-22158

Browse the full library · About the Tower of Babel