Toward Ultra-Long-Horizon Sequential Model Editing

Mingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa · 2026

Read Toward Ultra-Long-Horizon Sequential Model Editing by Mingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02543.txt.

Description

Mingda Liu and 3 co-authors diagnose why editing a language model's facts many times in a row eventually breaks it, in a 2026 arXiv paper. Sequential locate-and-edit methods can fail abruptly after many successful edits. The authors formalize the cause as a positive norm-feedback loop: solved value vectors and edited weights amplify each other until capabilities collapse.

Their analysis shows the loop can drive roughly exponential norm growth under standard editing dynamics. Existing per-edit regularizers and update clamps cannot resolve it. The proposed fix, norm-anchor scaling, rescales each solved value vector to a reference norm taken from the original model. The change is one line with negligible cost. Across multiple model backbones, datasets, and editors, it extended the usable editing horizon more than fourfold. Long-run editing performance improved by about seventy-two percent on average, while single-edit success stayed intact.

Catalog record

CREATORMingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02543

Browse the full library · About the Tower of Babel