GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs

Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li · 2026

Read GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs by Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08901.txt.

Description

Language models sometimes recite training text word for word, a privacy and copyright risk. Xuanqi Zhang and two co-authors show that blanket fixes hurt: most tokens behave normally, while a small minority carries most of the memorization signal. They locate those spots with a token-level likelihood ratio against a reference model less exposed to the tested passages.

From that insight they derive Gated Subspace Steering, a probe-and-steer fix that corrects only activations the probe flags. Its best form has a closed-form solution from a gradient-weighted activation matrix. Across four benchmarks it matches or beats the best memorization cuts while using ten to one hundred times less compute than optimization-based rivals.

Catalog record

CREATORXuanqi Zhang, Haoyang Shang, Xiaoxiao Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08901

Browse the full library · About the Tower of Babel