On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference
Zhengyi Li, Yakai Wang, Kang Yang, Yu Yu, Jiaping Gui, Yu Feng, Ning Liu, Minyi Guo, Jingwen Leng · 2026
Read On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference by Zhengyi Li, Yakai Wang, Kang Yang, Yu Yu, Jiaping Gui, Yu Feng, Ning Liu, Minyi Guo, Jingwen Leng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04901.txt.
Description
Zhengyi Li and eight co-authors show that a shuffling defence used in secure transformer inference is weaker than previously claimed. Secure inference should let a client learn only the final output while the server learns nothing about the input. Revealing middle activations speeds things up, but shuffling them was meant to stop clients from stealing model weights.
The authors attack the defence by aligning differently shuffled activations to one common ordering, then using them to extract weights. On Pythia and GPT-2 models the alignment errors were tiny, and for about a dollar of queries the recovered weights came very close to the true ones. The result urges caution about this efficiency shortcut.
Catalog record
| CREATOR | Zhengyi Li, Yakai Wang, Kang Yang, Yu Yu, Jiaping Gui, Yu Feng, Ning Liu, Minyi Guo, Jingwen Leng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04901