Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning

Jihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected] · 2026

Read Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning by Jihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20730.txt.

Description

Jihoon Kwon, Jiwon Choi, and Jy-yong Sohn propose distributional alignment as a design criterion for task vectors in in-context learning. Task vectors compress demonstrations into compact hidden-state forms, but quality has been judged only by downstream accuracy. The authors argue that task-vector inference should match in-context learning's predictive distribution. Their d_NTP metric measures the gap in next-token probabilities between the two modes, and it correlates strongly and negatively with downstream accuracy.

Catalog record

CREATORJihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74494
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20730

Browse the full library · About the Tower of Babel