Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering
Dongxuan ZhuThe Chinese University of Hong Kong [email protected], Ly Tran Ho KhanhThe Chinese University of Hong Kong [email protected], Andy Yat-Ming CheungHong Kong Polytechnic University [email protected] Man-Chung YueHong Kong Polytechnic University [email protected] V · 2026
Read Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering by Dongxuan ZhuThe Chinese University of Hong Kong [email protected], Ly Tran Ho KhanhThe Chinese University of Hong Kong [email protected], Andy Yat-Ming CheungHong Kong Polytechnic University [email protected] Man-Chung YueHong Kong Polytechnic University [email protected] V (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22010.txt.
Description
Dongxuan Zhu and co-authors help language models give more varied answers. Models often default to a narrow set of responses, even when many good answers exist. That sameness limits brainstorming, tutoring, and any tool where diversity of output is the point.
Their 2026 preprint explores diverse generation paths through activation steering done at inference time. Steering nudges the model's internal activations, here using a construction tied to the Stiefel manifold, a space of orthonormal frames. The aim is diversity without retraining and without wrecking quality. Practical steering methods matter for a simple reason. They can be applied to deployed models today. Users get several genuinely different drafts. They do not get near-copies of one safe answer.
Catalog record
| CREATOR | Dongxuan ZhuThe Chinese University of Hong Kong [email protected], Ly Tran Ho KhanhThe Chinese University of Hong Kong [email protected], Andy Yat-Ming CheungHong Kong Polytechnic University [email protected] Man-Chung YueHong Kong Polytechnic University [email protected] V |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22010