Beyond Linear Steering: Unified Multi-Attribute Control for Language Models

Narmeen Oozeer, Luke Marks, Shreyans Jain, Fazl Barez, Amirali Abdullah · 2025

Read Beyond Linear Steering: Unified Multi-Attribute Control for Language Models by Narmeen Oozeer, Luke Marks, Shreyans Jain, Fazl Barez, Amirali Abdullah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24535.txt.

Introduction

Narmeen Oozeer and four co-authors present a 2025 method to steer several traits of a language model at once. Traits such as tone and stance can clash when steered one by one. Classic steering adds fixed vectors and needs tuning per trait, which scales poorly. Their method trains one classifier on the model's hidden states instead.

At use time, slopes from that classifier give a fresh steering direction for any trait mix. No stored vectors or retraining are needed. Two new test sets probe mixed-trait control in tone and debate text. Across three model families, the method beat strong rivals as judged by both probe classifiers and model judges.

Catalog record

CREATORNarmeen Oozeer, Luke Marks, Shreyans Jain, Fazl Barez, Amirali Abdullah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE167 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24535

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Linear Steering: Unified Multi-Attribute Control for Language Models Narmeen Oozeer, Luke Marks, Shreyans Jain, Fazl Barez, Amirali Abdullah Abstract Controlling multiple behavioral attributes in large language models (LLMs) at inference time is a challenging problem due to interference between attributes and the limitations of linear steering methods, which assume additive behavior in activation space and require per-attribute tuning. We introduce K-Steering, a unified and flexible approach that trains a single non-linear multi-label classifier on hidden activations and computes intervention directions via gradients at inference time. This avoids linearity assumptions, removes the need for storing and tuning separate attribute vectors, and allows dynamic composition of behaviors without retraining. To evaluate our method, we propose two new benchmarks, ToneBank and DebateMix , targeting compositional behavioral control. Empirical results across 3 model families, validated by both activation-based classifiers and LLM-based judges, demonstrate that K-Steering outperforms strong baselines in accurately steering multiple behaviors. The implementation of the K-steering framework is available in our public repository 1 1 1 Code available at: https://github.com/withmartian/k-steering 1 Introduction Large language models (LLMs) are increasingly deployed in high-stakes settings, necessitating reliable mechanisms to guide their behavior at inference time. While alignment methods such as Reinforcement Learning from Human Feedback (RLHF) ( Ouyang et al., 2022 ) have improved overall model helpfulness and harmlessness, they are expensive and inflexible to deploy ( Wu et al., 2024 ) , may inadvertently make models less safe ( Qi et al., 2023 ) and cause model forgetting ( Luo et al., 2023 ) . In contrast, activation-based steering methods offer lightweight alternatives that modify internal representations at inference time to influence attributes such as tone, factuality, or toxicity without retraining the full model ( Wu et al., 2025 ; Dathathri et al., 2019 ; Xue et al., 2023 ) . Most existing approaches focus on single-attribute control, typically using linear interventions. These include difference-in-classes methods Bolukbasi et al. (2016) ; Dev and Phillips (2019) ; Marks and Tegmark (2023) ; Arditi et al. (2024) , vector-based edits Meng et al. (2022) , and gradient-guided control using small auxiliary classifiers ( Dathathri et al., 2019 ) . Extending such techniques to multi-attribute control is non-trivial; for instance, a naive extension by simply combining steering vectors often leads to interference or degraded fluency ( van der Weij et al., 2024 ; Nguyen et al., 2025 ; Song et al., 2025 ) , as well as separate hyperparameter sweeps required for computing each individual vector. More recent frameworks like MAT-STEER ( Nguyen et al., 2025 ) and CONFST ( Song et al., 2025 ) attempt to address this through orthogonality constraints and confident vector selection. However, they still assume an additive linear structure of the steering vectors and require per-attribute gating functions to be learnt on the activations at each token. In this paper, we introduce K-Steering , a method that extends non-linear gradient-based steering approaches ( Kirch et al., 2024 ) from single-attribute control to multi-attribute control settings. K-Steering fills an important gap in current methods by enabling simultaneous control over multiple behaviors using a single, unified classifier. Unlike prior approaches that steer each attribute independently, K-Steering learns a joint representation of multiple behaviors by training a classifier on hidden activations. At inference time, it uses the classifier’s gradients to adjust the model’s behavior toward (or away from) a specified set of target attributes. This approach avoids the assumption that behaviors compose linearly, handles inter-attribute interactions more robustly, and scales naturally to larger attribute sets. Empirically, we find that an activation classifier trained at the final layer detects a stronger presence of target class behaviors in K-Steered outputs compared to baselines that average independent steering vectors. These results are supported by traditional steering evaluation methods, such as LLM judges, which indicate that K-Steering better resists the dilution effect of averaging multiple vectors. Our contributions are three-fold: 1. Datasets : We introduce two novel datasets specifically designed for multi-attribute steering with K-Steering: TONEBANK for controlling conversational tones and DEBATEMIX for steering across different debate styles. 2. Methodology : We demonstrate that K-Steering effectively steers across multiple directions simultaneously, performing on par with or better than established baselines. By using additional gradient steps with smaller step sizes, we can smoothly improve steering performance. However, for larger step sizes, multi-step interventions can lead to degraded outputs. We also show that K-Steering directions can be ablated to remove behaviors, analogous to difference-in-means vector ablation as used by Arditi et al. (2024) . 3. Evaluation Framework : We develop a semi-automated evaluation protocol that calibrates steering magnitudes for both K-Steering and baseline methods. This protocol uses binary search to find the optimal magnitudes that maximize steering effectiveness while minimizing distribution shift. The remainder of the paper is structured as follows. Section 2 gives an overview of existing steering techniques and their shortcomings. Section 3 introduces our proposed method, K-Steering, including its formalization and gradient-based intervention algorithms. Section 4 describes the datasets and model configurations used in our experiments, with dataset construction details and representative examples provided in Appendices E and G , respectively. We also give a list of the attributes we use as steering targets for each dataset in Appendix F . In Section 5 , we compare K-Steering to baselines including CAA and DCT; details of the LLM-based evaluation protocol are provided in Appendix J and Appendix K . Section 6 analyzes K-Steering hyperparameters, including multi-layer and multi-step steering, with computational cost estimates included in Appendix C . Section 7 evaluates projection removal (Algorithm 2 ) as an efficient, one-step ablation method. Appendix H contains representative samples of K-Steered generation examples. Section 8 concludes with broader implications and future directions. 2 Background and Related Work 2.1 Language Model Alignment Language model alignment refers to the process of conforming model outputs with human preferences and values. Current alignment techniques include Reinforcement Learning from Human Feedback (RLHF) ( Ouyang et al., 2022 ; Bai et al., 2022a ; Yu et al., 2024 ; Dai et al., 2023 ) , Constitutional AI ( Bai et al., 2022b ; Huang et al., 2024 ) , and various forms of instruction tuning. While these methods have been successful in improving model behavior, they typically operate on a single dimension (e.g., helpfulness or harmlessness), and may encode alignment constraints in ways that are vulnerable to manipulation. 2.2 Activation Steering and Editing Recent work has explored manipulating language model behavior by directly intervening in the model’s activation space. Zou et al. (2023) , Li et al. (2023b) , and Turner et al. (2023) demonstrated that by identifying directions in the activation space corresponding to specific attributes, one can steer model generations by adding or removing projections along these directions. However, the most popular methods only focus on single-attribute interventions, and compute a steering vector using first-order, linear methods on hidden states. For example, "inference-time-intervention" (ITI) ( Li et al., 2023b ) learns a logistic regression model on the hidden states of positive vs negative examples, while "representation engineering" (RepE) ( Zou et al., 2023 ) extracts the first principal component of the difference between embeddings of positive and negative examples, and contrastive activation addition (CAA) ( Panickssery et al., 2023 ) returns the difference-of-means

Read the complete text