Activation Steering in Neural Theorem Provers
Shashank Kirtania · 2025
Read Activation Steering in Neural Theorem Provers by Shashank Kirtania (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15507.txt.
Introduction
Shashank Kirtania and Arun Iyer steer a language model while it writes formal proofs, in a 2025 paper whose text carries the title Steering LLMs for Formal Theorem Proving. Turning loose mathematical ideas into strict proofs is hard for these models.
The authors use a method called activation steering. They find directions inside the model's activity that match informal reasoning traces. Nudging the activity along those directions during writing improves the proofs, with no retraining. The trick worked under two ways of sampling answers. It also shows how the model stores reasoning cues inside itself. The study offers both a practical tool and a window into how proof writing happens inside the model.
Catalog record
| CREATOR | Shashank Kirtania |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15507
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Activation Steering in Neural Theorem Provers Shashank Kirtania Abstract Large Language Models (LLMs) have shown promise in proving formal theorems using proof assistants like Lean. However, current state of the art language models struggle to predict next step in proofs leading practitioners to use different sampling techniques to improve LLMs capabilities. We observe that the LLM is capable of predicting the correct tactic; however, it faces challenges in ranking it appropriately within the set of candidate tactics, affecting the overall selection process. To overcome this hurdle we use activation steering to guide LLMs responses to improve the generations at the time of inference. Our results suggest that activation steering offers a promising lightweight alternative to specialized fine-tuning for enhancing theorem proving capabilities in LLMs, particularly valuable in resource-constrained environments. 1 Introduction Interactive proof assistants such as Lean de Moura et al. 2015 , Isabelle Wenzel et al. 2008 , and Coq Barras et al. 1999 enable the formal verification of mathematical proofs and software by leveraging specialized programming languages Avigad 2023 ; Ringer et al. 2019 . Neural theorem proving, which integrates neural language models with interactive proof assistants, has emerged as a promising approach to automating formal reasoning First et al. 2023 ; Polu & Sutskever 2020b ; Polu et al. 2022 ; Yang et al. 2023b ; Welleck 2023 . This integration is mutually beneficial: proof assistants enforce formal correctness, while language models assist in proof construction by predicting and suggesting logical steps. A central challenge in this setting is tactic prediction—determining the appropriate next step at each proof state. In this work, we investigate activation steering Panickssery et al. 2024 ; Turner et al. 2024 ; Lucchetti & Guha 2024 as a technique to enhance tactic prediction in Llemma Azerbayev et al. 2024a and InternLM2 Ying et al. 2024a . These language models are designed for theorem proving by training and fine-tuning on mathematical data. Activation steering is an inference-time model editing method that modifies a model’s internal representations to guide its behavior toward desired outputs. We propose its application in refining tactic selection, aiming to improve both the accuracy and interpretability of proof automation. By systematically influencing LLMs’ reasoning process, our approach enables structured interventions that enhance model-driven theorem proving, leading to more reliable and controllable predictions. We present an approach for steering tactic selection from a pair of prompts (p_{1},p_{2}) that contain a LEAN state s to generate the next step (or tactic) t . However, the LLM successfully predicts t for p_{1} but mispredicts for p_{2} . In each pair p_{2} is natural data and p_{1} is synthetically generated using p_{2} . We systematically add the attributes and a high level of structure the proof should follow. These additional attributes guide the model to a specific and more grounded chain of thought to follow while predicting the tactics. These abstractions over the proof help the LLM to do critical decision making while predicting the next step. 2 Related Work 2.1 Formal Theorem Proving Formal theorem proving encodes theorems and proofs in a machine-verifiable format, ensuring correctness through rigid logical rules. A key component of this field is Interactive Theorem Proving (ITP), where humans collaborate with proof assistants such as Isabelle Wenzel et al. 2008 , Lean de Moura et al. 2015 , and Coq Barras et al. 1999 to formally verify proofs. These assistants allow users to express theorems in higher-order logic and construct verifiable proofs. In Lean de Moura et al. 2015 , proofs are built using tactics , which either solve a goal or decompose it into sub-goals. 2.2 Proofstep Generation with Large Language Models Generating intermediate proof steps is a fundamental challenge in theorem proving, particularly in tactic-based automated theorem provers (ATPs). Early neural approaches ( Whalen 2016 ; Huang et al. 2019 ; Bansal et al. 2019 ; Paliwal et al. 2020 ; Sanchez-Stern et al. 2020 ) framed proofstep generation as a classification task, employing models like TreeLSTM and RNN to predict tactics and their arguments. ASTactic ( Yang & Deng 2019 ) later introduced a grammar-constrained decoder for structured tactic generation. Recent advances leverage large language models (LLMs) for proof generation, casting tactic prediction as an extitauto-regressive sequence modeling problem. GPT- f ( Polu & Sutskever 2020a ) pioneered this approach by training transformers only with decoders to generate structured proof steps. Baldur ( First et al. 2023 ) extended this by producing entire proofs, while POETRY ( Wang et al. 2024 ) adopted a recursive decomposition strategy. Other works ( Szegedy et al. 2021 ; Tworkowski et al. 2022 ; Welleck et al. 2022 ; Jiang et al. 2022 ; Yang et al. 2023a ) integrate the selection of premises with tactic prediction, employing retrieval-augmented methods and constrained decoding to improve the coherence of the proof. Lean-Star Lin et al. 2024 introduced Self-Taught Reasoning, incorporating Chain-of-Thought Wei et al. 2023 reasoning before each tactic to generate synthetic data for fine-tuning LLMs via self-play Chen et al. 2024 . Our work leverages these randomly sampled data points from Lean-STaR-base as natural data. 2.3 Mechanistic Interpretability Previous work has focused on localizing and editing factual associations within transformers ( Meng et al. 2022 ) and probing hidden representations for high-level knowledge ( Li et al. 2024b ; Dong et al. 2023 ) . Such studies perform implicit evaluations of model ability, complementing explicit benchmarks ( Dong et al. 2023 ) . A key technique in mechanistic interpretability is activation patching ( Vig et al. 2020 ; Variengien & Winsor 2023 ) , which modifies model activations to influence outputs. This research has suggested the existence of task vectors ( Hendel et al. 2023 ; Ilharco et al. 2022 ) —representations encoding abstract task information. Activation steering has been employed to mitigate model deceitfulness and sycophancy ( Rimsky et al. 2023 ; Li et al. 2024b ) , further supporting the presence of task vectors. Steering is based on the linear representation hypothesis ( Park et al. 2023 ) , which posits that concepts exist as directions in the embedding space of the model. For theorem proving, mechanistic interpretability provides insights into how LLMs represent logical structures and reasoning processes. By dissecting these representations, we can identify failure cases, refine tactic prediction, and enhance proof generation. We hypothesize that effective steering transforms activations to align the model’s reasoning trajectory with a more structured and verifiable direction. 2.4 Model Steering Activation-based interventions can directly influence the language model output during inference Dathathri et al. 2019 ; Subramani et al. 2022 . Recent studies demonstrate that activation steering enhances truthfulness ( ) , mitigates sycophancy, and improves instruction-following Stolfo et al. 2024 , as well as type prediction in code Lucchetti & Guha 2024 . Building on these ideas, we investigate steering vectors in the context of theorem proving. Following prior work Burns et al. 2024 ; Turner et al. 2024 ; Arditi et al. 2024 ; van der Weij et al. 2024 , we compute steering vectors based on input pairs differing by a specific feature—here, the presence or absence of synthetic metadata. Unlike previous studies that focused on broad linguistic properties such as sentiment and style, our approach seeks to