AtPatch: Debugging Transformers via Hot-Fixing Over-Attention Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro,

Shihao Weng, Yang Feng, Jincheng Li, Yining Yin, Xiaofei Xie, Jia Liu · 2026

Read AtPatch: Debugging Transformers via Hot-Fixing Over-Attention Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, by Shihao Weng, Yang Feng, Jincheng Li, Yining Yin, Xiaofei Xie, Jia Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21695.txt.

Description

Shihao Weng, Yang Feng, Jincheng Li, Yining Yin, Xiaofei Xie, and Jia Liu fix transformer models by patching their attention. Models hit by backdoor attacks or bias often show odd attention patterns. They stare too hard at trigger tokens or at protected traits.

Older neuron-editing fixes are rigid and can warp the model's features. The authors hot-fix the over-attention directly, in the spirit of a software patch. A targeted patch should remove the bad behavior while leaving honest skills intact. The approach suits deployed models, where retraining from scratch is too slow when a flaw is found.

Catalog record

CREATORShihao Weng, Yang Feng, Jincheng Li, Yining Yin, Xiaofei Xie, Jia Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21695

Browse the full library · About the Tower of Babel