Steering LLM Reasoning Through Bias-Only Adaptation
Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov · 2025
Read Steering LLM Reasoning Through Bias-Only Adaptation by Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18706.txt.
Introduction
Viacheslav Sinii and co-authors show, in a 2025 paper, that a handful of steering vectors can teach a language model to reason as well as a full fine-tune. Their method trains one small vector per layer with reinforcement learning while every base weight stays frozen. On an eight billion parameter model, the added parts amount to a tiny sliver of extra size.
Across several base models and math reasoning tests, the light method matched fully tuned models. The savings are practical: less memory for the optimizer and less traffic between processors, which cuts the cost of training. A look inside the model shows the vectors amplify word directions that hang together, which sharpens the chain of thought. The authors argue that high-level reasoning needs far fewer trained numbers than assumed.
Catalog record
| CREATOR | Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18706
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Steering LLM Reasoning Through Bias-Only Adaptation Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov Abstract Recent work on reasoning‐oriented language models, exemplified by o1-like systems, suggests that reinforcement-learning (RL) finetuning does not create new capabilities but instead strengthens reasoning patterns already latent in the pretrained network. We test this claim by training steering vectors : layer-wise biases that additively amplify selected hidden features while leaving all original weights unchanged. Experiments on four base models across the GSM8K and MATH benchmarks show that steering vectors recover, and in several cases exceed, the accuracy of fully-tuned counterparts. This result supports the view that the required reasoning skills pre-exist in the base model. Further, logit-lens analysis reveals that the trained vectors consistently boost token groups linked to structured languages and logical connectors, providing an interpretable account that aligns with the demands of quantitative reasoning tasks. 1 Introduction Table 1: \text{mean}@8 accuracy for each combination of training dataset, evaluation dataset, model, and tuning setup. Rows are grouped by Train / Test dataset pairs, and each column corresponds to a specific model variant. For Steering and LoRA rows, the colored value in parentheses indicates the difference compared to Full-Tune for that model - green if better, red if worse. This highlights how close lightweight methods can get to full fine-tuning performance, and where gaps remain. Train / Test Setup Qwen2.5-1.5B Qwen2.5-Math-1.5B Llama3.1-8B Llama3.1-8B-It GSM8K / GSM8K Base 0.63 29.26 1.08 66.03 Full-Tune 78.91 86.49 76.49 87.22 Steering 73.84 (-5.07) 79.89 (-6.60) 70.36 (-6.13) 87.22 (0.00) LoRA 76.49 (-2.42) 85.41 (-1.08) 74.24 (-2.25) 85.41 (-1.81) GSM8K / MATH Base 1.51 28.73 0.76 32.51 Full-Tune 40.78 65.95 16.86 46.65 Steering 48.69 (+7.91) 61.79 (-4.16) 22.81 (+5.95) 48.51 (+1.86) LoRA 49.32 (+8.54) 64.31 (-1.64) 19.28 (+2.42) 49.04 (+2.39) MATH / MATH Base 1.51 28.73 0.76 32.51 Full-Tune 44.48 70.44 27.39 52.39 Steering 51.39 (+6.91) 65.27 (-5.17) 22.05 (-5.34) 50.81 (-1.58) LoRA 53.68 (+9.20) 69.35 (-1.09) 24.32 (-3.07) 50.40 (-1.99) MATH / GSM8K Base 0.63 29.26 1.08 66.03 Full-Tune 68.57 82.56 52.12 85.03 Steering 69.56 (+0.99) 76.41 (-6.15) 45.24 (-6.88) 84.02 (-1.01) LoRA 72.53 (+3.96) 81.88 (-0.68) 52.52 (+0.40) 85.06 (+0.03) Large language models (LLMs) such as GPT-4 ( Achiam et al. 2023 ) , Qwen ( Yang et al. 2025 ) , and Llama ( AI@Meta 2024 ) demonstrate that scaling up data, model size, and compute can create a general-purpose engine for language understanding and generation. Trained on trillions of tokens from books, code, scientific articles, and social media, these models can summarize long documents, write executable code, solve domain-specific exams, and generate persuasive prose - all without task-specific tuning. Their success has shifted the research focus from whether large-scale pretraining works to how to elicit the rich behaviours embedded within, ranging from chain-of-thought reasoning to tool use and self-correction. Recent systems such as openai-o1 and R1 ( Guo et al. 2025 ) fine-tune these base models with reinforcement learning (RL) to create ”reasoning” models. The authors of these systems argue that RL training imparts new abilities (such as self-reflection and sudden ”aha” moments) that purportedly emerge only after RL. However, growing evidence ( Liu et al. 2025 ; Shah et al. 2025 ) points to the contrary. The studies indicate that the necessary skills are already present in the pretrained network, with RL serving primarily to amplify, not invent, them. We directly test this claim by training steering vectors : simple, trainable vectors that are added to the model’s hidden state at each layer, leaving all original weights untouched. If reasoning ability is already latent in the model, steering in the right direction should suffice to unlock it (see Section 2 for further details). We train steering vectors on four base models ( Qwen-2.5-1.5B , Qwen-2.5-Math-1.5B , Llama-3.1-8B , and Llama-3.1-8B-Instruct ) using two mathematical reasoning datasets ( GSM8K and MATH ) ( Cobbe et al. 2021 ; Hendrycks et al. 2021 ) . Across these eight model-task pairs, we find that steering alone often matches the accuracy of fully RL-tuned models, and in all cases performs competitively. These findings support the hypothesis that LLMs already possess the knowledge necessary for step-by-step reasoning, which can be elicited via targeted amplification. 2 Related Work The use of steering vectors, a technique within activation engineering, provides a direct way to probe and manipulate model behavior with minimal changes to the underlying weights. Traditionally, such vectors are constructed from activation differences on contrastive prompts (e.g., positive vs. negative sentiment) and are typically interpreted as feature amplifiers rather than creators of novel behaviors ( Turner et al. 2023 ; Panickssery et al. 2023 ) . More recent work demonstrates that these vectors can also be trained, not merely computed, allowing for more targeted control. For example, Cao et al. 2024 optimized steering directions using preference data, while Mack & Turner 2024 and Engels et al. 2025 (building on Betley et al. 2025 ) showed that training simple additive vectors can elicit complex latent behaviors, such as reasoning and self-awareness. Our method follows this trend, training only layer-wise additive biases while keeping all other model parameters frozen. This approach is philosophically aligned with BitFit ( Zaken et al. 2021 ) , which tunes only bias terms and has been shown to effectively expose existing knowledge, often matching the performance of full fine-tuning on language tasks. Notably, BitFit and similar minimal-adaptation methods sometimes underperform on tasks requiring substantial generalization ( Hu et al. 2022 ) ; it remains unclear whether they suffice for complex reasoning. These minimal interventions stand in contrast to parameter-efficient finetuning methods such as prompt tuning and LoRA ( Hu et al. 2022 ) , or full RL-based adaptation ( Guo et al. 2025 ) , which actively adjust model parameters. In our work, we focus on whether steering vectors alone can unlock reasoning capabilities already present in LLMs, without the need for extensive weight updates. Table 2: Clusters of tokens most aligned with the learned steering vectors, as measured by cosine similarity. Layer idx Top-Cluster Representative tokens Unifying idea 2 Source-code & test-harness vocabulary tostring , ComponentFixture , .SQL , standalone , -independent , _fault , 203 , @a , \Context These are the words you meet in programming projects - Angular’s ComponentFixture, SQL file extensions, ”fault” flags, HTTP status 203, context objects, and helper functions like toString(). 2 Named entities (people & places) Antonio , Pelosi , Baldwin , Cumberland , Switzerland , Peg , Salv- Proper names of individuals and locations that commonly co-occur in news articles or knowledge-graph dumps. 17 Accuracy, validation & logical necessity correctness , correct , precision , necessity , possibility , confirmation , answer , goal , directly , derive / deriving These words belong to discourse about getting things right - arguments, proofs, validations, QA reports, or formal specifications. 30 Causal & contrastive connectors Because / because / 因为 , Therefore / donc , However / однако / jedoch , Given / Here , step Words that introduce reasons, consequences, or contrasts - typical of argumentative writing, technical explanations, or test-case descriptions. 3 Methodology 3.1 Online Training We adopt an online reinforcement learning procedure loosely modeled on DeepSeek-R1 ( Ahmadian et al. 2024 ; Guo et