Toward Ultra-Long-Horizon Sequential Model Editing
Mingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa · 2026
Read Toward Ultra-Long-Horizon Sequential Model Editing by Mingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02543.txt.
Description
Mingda Liu and 3 co-authors diagnose why editing a language model's facts many times in a row eventually breaks it, in a 2026 arXiv paper. Sequential locate-and-edit methods can fail abruptly after many successful edits. The authors formalize the cause as a positive norm-feedback loop: solved value vectors and edited weights amplify each other until capabilities collapse.
Their analysis shows the loop can drive roughly exponential norm growth under standard editing dynamics. Existing per-edit regularizers and update clamps cannot resolve it. The proposed fix, norm-anchor scaling, rescales each solved value vector to a reference norm taken from the original model. The change is one line with negligible cost. Across multiple model backbones, datasets, and editors, it extended the usable editing horizon more than fourfold. Long-run editing performance improved by about seventy-two percent on average, while single-edit success stayed intact.
Catalog record
| CREATOR | Mingda Liu, Zhenghan Zhu, Ze’an Miao, Katsuki Fujisawa |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 109 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02543