Understanding the Dynamics of Demonstration Conflict in In-Context Learning
Difan Jiao, Di Wang, Lijie Hu · 2026
Read Understanding the Dynamics of Demonstration Conflict in In-Context Learning by Difan Jiao, Di Wang, Lijie Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04464.txt.
Description
Jiao and colleagues trace how conflicting examples mislead in-context learning. Few shot examples let models infer rules, but noisy examples make that skill fragile. One corrupted rule example causes large degradation. Probes show models encode both correct and incorrect rules early, gaining confidence only late. Early vulnerability heads show positional bias, while late susceptible heads cut support for correct predictions. Masking a few heads improves performance by over 10 percent.
Catalog record
| CREATOR | Difan Jiao, Di Wang, Lijie Hu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52078 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.04464