Understanding the Dynamics of Demonstration Conflict in In-Context Learning

Difan Jiao, Di Wang, Lijie Hu · 2026

Read Understanding the Dynamics of Demonstration Conflict in In-Context Learning by Difan Jiao, Di Wang, Lijie Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04464.txt.

Description

Jiao and colleagues trace how conflicting examples mislead in-context learning. Few shot examples let models infer rules, but noisy examples make that skill fragile. One corrupted rule example causes large degradation. Probes show models encode both correct and incorrect rules early, gaining confidence only late. Early vulnerability heads show positional bias, while late susceptible heads cut support for correct predictions. Masking a few heads improves performance by over 10 percent.

Catalog record

CREATORDifan Jiao, Di Wang, Lijie Hu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52078
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.04464

Browse the full library · About the Tower of Babel