Uncovering Context Reliance in Unstructured Knowledge Editing
Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren · 2026
Read Uncovering Context Reliance in Unstructured Knowledge Editing by Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19043.txt.
Description
Editing a language model with messy real-world text often fails in a subtle way. Zisheng Zhou, with six co-authors, names that failure Context Reliance in 2026. Knowledge learned from edited text sticks to its preceding context. When that context vanishes at inference time, recall fails. Prepending context restores recall, confirming the diagnosis.
The authors show the flaw is built into gradient-based training. It binds new knowledge to one aggregated context view. Their fix is COIN, a context-independent editing frame. It pushes the model to focus on knowledge in local scope, not on context patterns. Tests show COIN cuts Context Reliance sharply. It also beats strong baselines by a wide margin on editing success. Mitigating Context Reliance thus matters for robust editing.
Catalog record
| CREATOR | Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-19043