Uncovering Context Reliance in Unstructured Knowledge Editing

Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren · 2026

Read Uncovering Context Reliance in Unstructured Knowledge Editing by Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19043.txt.

Description

Editing a language model with messy real-world text often fails in a subtle way. Zisheng Zhou, with six co-authors, names that failure Context Reliance in 2026. Knowledge learned from edited text sticks to its preceding context. When that context vanishes at inference time, recall fails. Prepending context restores recall, confirming the diagnosis.

The authors show the flaw is built into gradient-based training. It binds new knowledge to one aggregated context view. Their fix is COIN, a context-independent editing frame. It pushes the model to focus on knowledge in local scope, not on context patterns. Tests show COIN cuts Context Reliance sharply. It also beats strong baselines by a wide margin on editing success. Mitigating Context Reliance thus matters for robust editing.

Catalog record

CREATORZisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19043

Browse the full library · About the Tower of Babel