UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering

Yingdong Shi, Ruiming Zhang, Changming Li, Zhiyu Yang, Kaixing Zhang, Jingyi Yu, Kan Ren ShanghaiTech University {shiyd2023, zhangrm2022, renkan}@shanghaitech.edu.cn · 2026

Read UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering by Yingdong Shi, Ruiming Zhang, Changming Li, Zhiyu Yang, Kaixing Zhang, Jingyi Yu, Kan Ren ShanghaiTech University {shiyd2023, zhangrm2022, renkan}@shanghaitech.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30076.txt.

Introduction

Yingdong Shi and six colleagues present UniSteer, a text-guided way to steer language models by shaping inner activations. Older tricks use fixed directions or one-off modules, weak on fine-grained ideas and combined demands. UniSteer learns one conditional flow over residual-stream activations from plain-language conditions, then moves and rebuilds a source activation under a target instruction. The same model classifies by picking the label with the lowest rebuild cost. Tests on three models cover behaviour, truthfulness, fine concepts and combined instructions.

Catalog record

CREATORYingdong Shi, Ruiming Zhang, Changming Li, Zhiyu Yang, Kaixing Zhang, Jingyi Yu, Kan Ren ShanghaiTech University {shiyd2023, zhangrm2022, renkan}@shanghaitech.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53475
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30076

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering Yingdong Shi, Ruiming Zhang, Changming Li, Zhiyu Yang, Kaixing Zhang, Jingyi Yu, Kan Ren ShanghaiTech University {shiyd2023, zhangrm2022, renkan}@shanghaitech.edu.cn Abstract Activation-based control steers large language models (LLMs) by intervening on their internal representations during inference, and has emerged as an effective paradigm for controlling behaviors such as persona and style. However, existing methods often rely on fixed steering directions or task-specific intervention modules, making them difficult to adapt to fine-grained concepts and compositional constraints. We propose UniSteer , a text-guided activation flow matching model that learns a conditional distribution over residual-stream activations from natural-language conditions. Instead of fitting a separate intervention for each target behavior, UniSteer learns a universal conditional velocity field in activation space. At inference time, UniSteer performs flow inversion by partially transporting a source activation toward a latent state and regenerating it under a target textual condition before injecting it back into the frozen LLM. The same conditional model supports activation-space classification by selecting the textual label with the lowest reconstruction energy. Experiments on three target LLMs show that UniSteer provides a unified interface across behavioral control, truthfulness steering, fine-grained concept steering, multi-constraint instruction following, and activation-space classification. 1 Introduction Controlling the behavior of large language models (LLMs) is central to their safe, reliable, and customizable deployment. One promising direction is activation-based control, which intervenes directly on the internal representations of a frozen LLM during inference ( 29 ; 24 ; 16 ; 35 ) . Compared with prompting or fine-tuning, activation intervention offers a lightweight and modular way to influence model behavior without updating model parameters, making it attractive for steering properties such as truthfulness, refusal, persona, style, and instruction following. Existing activation steering methods typically represent a target behavior as a fixed direction or task-specific intervention in activation space. Contrastive activation addition ( 24 ; 29 ) estimates steering vectors from positive and negative examples, representation engineering ( 35 ) identifies behavior-relevant directions or subspaces, and learned intervention methods ( 33 ; 34 ; 20 ) train modules to modify hidden states. Although effective in several settings, these approaches are often tied to predefined attributes, require separately fitted directions or modules for each target behavior, and struggle to compose multiple behavioral requirements because independently learned directions can interfere with one another in high-dimensional activation spaces. We argue that activation steering can be more naturally formulated through text-conditioned activation flow matching ( 18 ; 19 ; 28 ) . Rather than constructing a separate intervention for each target behavior, the goal is to learn a conditional velocity field over LLM activations, where the editing dynamics are specified by a semantic text condition. This view provides a unified interface for heterogeneous control targets including behavioral traits, fine-grained concepts, and multi-constraint requirements. The control targets can all be expressed as textual conditions, while the same activation model defines the corresponding editing dynamics. In particular, compositional requirements can be represented directly in the condition text, avoiding post-hoc combinations of separately learned steering components. In this work, we propose UniSteer , a text-conditioned activation flow model for unified LLM steering and activation-space classification ( 15 ; 5 ) . UniSteer learns a conditional velocity field over residual-stream activations of a frozen target LLM, where the condition is a natural-language description of the desired behavior, concept, or constraint. At inference time, UniSteer edits an observed activation by partially inverting it under a source condition and then transporting it forward under a target condition. The same conditional activation model can also be used as an activation-space classifier . Given candidate textual labels, UniSteer scores how well each condition explains an activation and predicts the label with the lowest conditional reconstruction energy. Our contributions are threefold. First, we formulate activation steering as text-conditioned activation transport and introduce a conditional flow-matching model for LLM internal activations. Second, we propose flow inversion for inference-time activation editing, enabling a single model to handle behavioral traits, fine-grained concepts, and compositional constraints through natural-language conditions. Third, we show that the same conditional activation model can be used for activation-space classification via reconstruction energy. 2 Related Work 2.1 Representation Understanding A growing body of work shows that the internal activations of large language models contain rich, structured information about model behavior. Linear probes and unsupervised representation methods ( 3 ; 1 ) have identified latent directions or subspaces associated with truthfulness, latent knowledge, factuality, refusal, spatial and temporal concepts, style, sentiment, subjective evaluation, and even task complexity ( 21 ; 11 ; 30 ; 27 ) . Beyond probing raw residual streams, sparse autoencoders extract more interpretable feature dictionaries from activations ( 6 ; 7 ) , and latent-space monitors can detect unsafe or deceptive behaviors from hidden states ( 10 ) . Collectively, the richness of this internal information provides a theoretical foundation for conditional activation steering. 2.2 Activation Steering Activation steering modifies LLM behavior by intervening on internal representations during generation. Most prior methods either construct fixed behavior directions from contrastive examples ( 24 ; 29 ; 35 ) or learn task-specific intervention modules ( 33 ; 34 ; 20 ) . Although effective, these methods usually require separately fitted directions or modules for each target behavior and can suffer from interference when multiple requirements are combined. UniSteer instead learns a natural-language-conditioned velocity field over activations, enabling a single model to handle single-behavior and compositional steering conditions. 2.3 Flow Matching for Editing and Conditional Classification Flow matching provides a continuous-time framework for high-dimensional generation ( 18 ; 19 ; 28 ) . Prior work has shown that generative flows and diffusion models support both editing through partial inversion or noising ( 22 ; 13 ; 23 ) and classification by comparing conditional reconstruction or likelihood scores ( 15 ; 5 ) . UniSteer transfers these properties from image generation to LLM activation spaces. 3 Methodology Figure 1: Overview of UniSteer . (a) During training, residual-stream activations are extracted from selected layers and token positions of a frozen language model and paired with natural-language conditions. A frozen condition model encodes the textual condition, and UniSteer learns a text-guided conditional flow in activation space via flow matching. (b) During inference, UniSteer performs activation steering through flow inversion. A source activation is first transported backward along the source-conditioned flow to an intermediate noisy latent state, and then transported forward under the target condition to obtain an edited activation. The edited activation is injected back into the frozen language model to steer generation. We propose UniSteer , a text-conditioned activation flow model for steering and classifying internal representations of a frozen language model. As shown in Figure 1 , UniSteer learns a conditional flow over residual-stream activations paired with natural-language conditions. During training, we extract residual-stream activations from selected layers of the frozen target model and pair them with natural-language conditions. A frozen condition model encodes the condition, and a conditional flow model is trained to transport noise to activations associated with the given condition. This yields a text-conditioned activation distribution over model internals. At inference time, it edits an observed activation by partially inverting it under a source condition and regenerating it under a target condition. The same model

Read the complete text