In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification

Ming Liu · 2026

Read In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification by Ming Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08295.txt.

Description

Ming Liu reports a sharp failure mode in few-shot prompting. Earlier work found that random labels in examples barely hurt classification. Liu shows a worse case. When every example carries the same label, accuracy collapses. It falls to twelve percent or less across six model families and four tasks. That holds even when the label is valid.

The cause sits in the label slot itself. Models treat the tokens in label spots as the whole allowed answer set. A single repeated label is the extreme case. Given varied nonsense labels, models still put most of their probability on the shown set. The author calls this set-level fixation. He uses it against Bayesian accounts of in-context learning.

Catalog record

CREATORMing Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08295

Browse the full library · About the Tower of Babel