GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs
Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li · 2026
Read GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs by Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08901.txt.
Description
Language models sometimes recite training text word for word, a privacy and copyright risk. Xuanqi Zhang and two co-authors show that blanket fixes hurt: most tokens behave normally, while a small minority carries most of the memorization signal. They locate those spots with a token-level likelihood ratio against a reference model less exposed to the tested passages.
From that insight they derive Gated Subspace Steering, a probe-and-steer fix that corrects only activations the probe flags. Its best form has a closed-form solution from a gradient-weighted activation matrix. Across four benchmarks it matches or beats the best memorization cuts while using ten to one hundred times less compute than optimization-based rivals.
Catalog record
| CREATOR | Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-08901