Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
Jihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected] · 2026
Read Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning by Jihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20730.txt.
Description
Jihoon Kwon, Jiwon Choi, and Jy-yong Sohn propose distributional alignment as a design criterion for task vectors in in-context learning. Task vectors compress demonstrations into compact hidden-state forms, but quality has been judged only by downstream accuracy. The authors argue that task-vector inference should match in-context learning's predictive distribution. Their d_NTP metric measures the gap in next-token probabilities between the two modes, and it correlates strongly and negatively with downstream accuracy.
Catalog record
| CREATOR | Jihoon KwonSeoul National University [email protected], Jiwon ChoiYonsei University [email protected], Jy-yong SohnYonsei University [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74494 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20730