ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization
Riyaz Ahuja, Tate Rowney, Jeremy Avigad, Sean Welleck · 2026
Read ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization by Riyaz Ahuja, Tate Rowney, Jeremy Avigad, Sean Welleck (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22885.txt.
Introduction
Ahuja and three colleagues present ImProver 2, a neurosymbolic framework for optimizing verified proofs in Lean 4, where libraries keep growing and hand refactoring lags. It pairs a data-efficient expert-iteration pipeline plus a scaffold that exposes formal structure beside light informal abstractions, plus metrics for structural proof traits. A 7 billion parameter model trained this way beat far larger models in its family and rivaled mid-tier frontier systems across metrics.
Catalog record
| CREATOR | Riyaz Ahuja, Tate Rowney, Jeremy Avigad, Sean Welleck |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 111367 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22885
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization Riyaz Ahuja, Tate Rowney, Jeremy Avigad, Sean Welleck Abstract Formal mathematics libraries are rapidly expanding, creating a growing need to refactor verified proofs for maintainability and to improve training data quality for neural provers. However, scalable proof optimization is hindered by heterogeneous and heuristically specified objectives, scarce data, and high training and inference costs. To overcome these challenges, we introduce ImProver 2, a neurosymbolic framework for automated proof optimization in Lean 4. ImProver 2 combines a data-efficient expert-iteration pipeline with a scaffold that exposes formal structure alongside lightweight informal abstractions. We further introduce a suite of metrics capturing structural proof properties. Using ImProver 2, we train a 7B-parameter model that outperforms orders-of-magnitude larger models within the same model family, and is competitive with mid-tier frontier models across metrics. We additionally demonstrate that our neurosymbolic scaffold significantly improves performance across both small and frontier models. We show that with proper scaffolding and training, small models can effectively restructure research-level proofs over complex and varied metrics, matching substantially larger systems and establishing proof optimization as a scalable, learnable task. 1 Introduction Formal proof assistants such as Lean ( 14 ) , Rocq ( 21 ) , and Isabelle ( 25 ) have transformed mathematical practice by making the correctness of proofs explicit and mechanized. Community libraries such as Lean’s Mathlib ( 22 ) now grow at a pace driven both by increasing human contributions and by recent advances in neural theorem proving and autoformalization ( 1 ; 9 ) . This rapid expansion raises multiple concerns about issues of data quality. First, this expansion stresses the maintainability, coherence, and long-term usability of libraries due to proofs that are often heterogeneous in style and clarity ( 22 ) . Second, low library quality decreases its utility as training data: modern theorem provers and autoformalizers increasingly train on these very corpora, so the structure and readability of proofs directly shape downstream prover performance ( 7 ) . Unfortunately, the growth rate of formal libraries already exceeds what human reviewers and maintainers can reliably curate; and moreover, this discrepancy is only expected to widen due to increasing volumes of machine-generated proofs which, even when guaranteed to be correct, do not carry similar guarantees as to the proof’s quality, modularity, or understandability ( 4 ) . This motivates automated proof optimization: given a verified proof, produce a formally correct rewrite that scores better under a user-defined objective, such as shorter length, higher modularity, or fewer explicit dependencies ( 2 ) . Because objectives vary by use case and formal context, a practical method must scale across arbitrary metrics and research-level theorems. Small specialized models are especially attractive for this setting because relevant data is scarce in general corpora, library-scale deployment can require millions of samples, and local open-weight models are more accessible to formalization projects. We address these problems with ImProver 2 , a self-improving pipeline that trains small language models ( SLM s) to optimize Lean proofs under a wide-ranging class of metrics. Our core idea is to leverage iterative preference optimization : iterating between generating proof candidates, scoring them according to correctness and the desired optimization criterion, and learning from the resulting pairs of higher and lower scoring proofs. In particular, we extend the Iterative Reasoning Preference Optimization (IRPO) ( 15 ) algorithm with a new replay buffer that balances old and newly generated data for use in the next round of training, preventing model collapse and allowing for monotonic improvement over many rounds. We additionally give the model access to rich information from the Lean theorem proving environment, including goal states, informalized summaries, lemma context, and examples, which we term neurosymbolic augmentation . We use ImProver 2 to train models for three different metrics: the length of the proof, modularity (the ability to divide the proof into a series of smaller lemmas), and the explicit dependencies used by the theorem, and show that our trained models can substantially improve over their base model and remain competitive with much larger unscaffolded systems on research-level theorems. In summary, our contributions are: 1. Self-improving SLMs for proof refactoring. We show that iterative preference optimization can bootstrap small language models on proof optimization in specialized research libraries, making them competitive with much larger models in several unscaffolded comparisons. 2. Structural optimization metrics. Beyond proof length ( 2 ; 7 ) , we study modularity and dependency metrics that leverage proof structure and the surrounding library. These metrics were chosen with formal mathematics experts and target distinct proof-refactoring objectives. 3. Neurosymbolic augmentation for research mathematics. We extract relevant lemmas or definitions, goal-state traces, and automatic informalizations of target proofs. This augmentation boosts both small and large models on proof optimization. We additionally open-source our code and data 1 1 1 Github . 2 Related Work Interest in neurosymbolic theorem proving–the use of deep learning to create or manipulate verified mathematical proofs in languages such as Lean 4 ( 14 ) –has seen significant advancements in recent years ( 13 ; 11 ) . In particular, much research has focused on generating formal proofs given their statements ( 16 ) , with recent systems achieving high performance on nontrivial benchmarks and internationally renowned mathematics competitions ( 9 ; 1 ; 4 ) . Many systems additionally utilize neurosymbolic augmentation, providing a generative prover model with information gathered from within the proof environment ( 27 ; 2 ; 12 ) . However, both formal and informal proofs generated by current LLM-based systems often suffer from stylistic irregularities even when they are sound, including redundant steps or a structure which does not clearly represent the broader logical argument ( 6 ) . Previous work by ( 2 ; 7 ) has attempted to rectify these issues by creating LLM-based agents to refactor formal proofs. 2 created a system capable of optimizing towards multiple metrics of improvement; however, it relied on general-purpose closed-source models, leading to substantial deployment costs and limited ability to improve performance beyond these models’ baseline. 7 focused solely on optimizing the token count of proofs according to a complex tokenizer intended to reduce the time required to compile them; they do not examine other metrics, leaving out important use cases and limiting its utility to research mathematicians. Furthermore, the works above do not fully utilize the information available through working in an interactive theorem proving environment via goal-state extraction ( 16 ) , premise retrieval ( 27 ) , or auto-informalization ( 8 ) ; both of the above overlook at least one of these aspects. 3 Proof Optimization Given a verified proof, a proof optimization agent synthesizes a semantically equivalent proof that is “better” under a user-specified objective while remaining correct according to the Lean kernel. We follow the setup of 2 ; 7 and introduce two additional structural metrics: modularity and dependencies . Original ⬇ theorem isCoatom_iff [ OrderTop A ] { K : A } : IsCoatom K ↔ K ≠ T ∧ ∀ H g , K ≤ H → g ∉ K → g ∈ H → H = T := by simp_rw [ IsCoatom , lt_iff_le_not_le , SetLike . not_le_iff_exists , and_comm (