How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning

Entang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn · 2026

Read How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning by Entang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16591.txt.

Description

Wang, Wang, Bakalova and Hahn ask how few-shot examples combine inside a language model to form its function vector, the direction that steers task behavior. They show an n-shot vector is well matched by adding example-level pieces. Yet context reweights those pieces toward clearer, more informative examples. A causal split finds Query-Key routing drives most of the gain, especially when examples are ambiguous, while Value effects vary more.

Catalog record

CREATOREntang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122252
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16591

Browse the full library · About the Tower of Babel