Unveiling and Eliminating the Shortcut Learning for Locate-Then-Edit Knowledge Editing via Both Subject and Relation Awareness

Xiyu Liu, Zhengxiao Liu, Naibin Gu, Zheng Lin, Ji Xiang, Weiping Wang · 2025

Read Unveiling and Eliminating the Shortcut Learning for Locate-Then-Edit Knowledge Editing via Both Subject and Relation Awareness by Xiyu Liu, Zhengxiao Liu, Naibin Gu, Zheng Lin, Ji Xiang, Weiping Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04042.txt.

Description

Liu and co-authors stop a shortcut that corrupts edits to model knowledge in a 2025 paper. Locate-then-edit methods update one fact inside a language model, but the edit often wrecks other knowledge about the same subject. Their diagnosis: the training objective leans on the subject and skips the relation that gives the fact its meaning.

Their fix, Causal Path Alignment, forces updates to travel through relation-aware middle states along valid causal paths. Across several model backbones the shortcut vanished, relation-specific accuracy rose, and side effects stayed small. The method also plugs into existing editors, aiming to make in-place knowledge updates trustworthy.

Catalog record

CREATORXiyu Liu, Zhengxiao Liu, Naibin Gu, Zheng Lin, Ji Xiang, Weiping Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04042

Browse the full library · About the Tower of Babel