TheoremForge: Scaling up Formal Data Synthesis with Low-Budget Agentic Workflow
Yicheng Tao, Hongteng Xu · 2026
Read TheoremForge: Scaling up Formal Data Synthesis with Low-Budget Agentic Workflow by Yicheng Tao, Hongteng Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17332.txt.
Introduction
Yicheng Tao and Hongteng Xu present TheoremForge, a low-cost pipeline to make formal math data. Formal data trains and tests proof tools. Agentic workflows can make it, but they cost too much. That cost keeps open corpora scarce. TheoremForge splits the work into five sub-tasks to cut waste. The steps run from statement formalization to proof correction.
The steps are statement writing, proof generation, premise choice, proof repair, and one more linked stage. Each step can use the cheapest tool that does it well. More data at low cost helps open proof research grow. It also lets small teams join a field ruled by big labs. The pipeline is a bid for scale with care.
Catalog record
| CREATOR | Yicheng Tao, Hongteng Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17332
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TheoremForge: Scaling up Formal Data Synthesis with Low-Budget Agentic Workflow Yicheng Tao, Hongteng Xu Abstract The high cost of agentic workflows in formal mathematics hinders large-scale data synthesis, exacerbating the scarcity of open-source corpora. To address this, we introduce TheoremForge , a cost-effective formal data synthesis pipeline that decomposes the formalization process into five sub-tasks, which are statement formalization , proof generation , premise selection , proof correction and proof sketching . By implementing a Decoupled Extraction Strategy , the workflow recovers valid training signals from globally failed trajectories, effectively utilizing wasted computation. Experiments on a 2,000-problem benchmark demonstrate that TheoremForge achieves a Verified Rate of 12.6%, surpassing the 8.6% baseline, at an average cost of only $0.481 per successful trajectory using Gemini-3-Flash. Crucially, our strategy increases data yield by 1.6 × \times for proof generation compared to standard filtering. These results establish TheoremForge as a scalable framework for constructing a data flywheel to train future expert models. Our code is available here . 1]Gaoling School of Artificial Intelligence, Renmin University of China 2]Beijing Key Laboratory of Research on Large Models and Intelligent Governance 3]Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE 1 Introduction Agentic workflows based on large language models have recently achieved significant advancements in the domain of formal mathematics. Compared to traditional single-turn interaction paradigms, agentic workflows demonstrate superior performance and robustness in autoformalization and automated theorem proving tasks ( 25 ; 4 ; 5 ; 34 ; 27 ) . This success is largely attributed to targeted strategy designs and test-time scaling techniques. However, the scarcity of training data hinders further development in this field. The diverse task requirements within workflows and the inherent data scarcity of the formalization domain exacerbate this challenge. Many current studies employ proprietary data for training while open-source formalization data remain insufficient in terms of quantity and diversity. Synthesizing training data suitable for formal reasoning models at a low cost and large scale has therefore become a critical research question. Current formalization tasks are primarily categorized into Autoformalization and Automated Theorem Proving. A complete formalization workflow should be capable of translating natural language mathematical propositions into semantically consistent formal propositions and generating compilable proofs. We analyze common and effective strategy designs in existing methods to extract five sub-tasks (see Figure 1 ). These include statement formalization ( 8 ; 3 ; 27 ) , proof generation ( 26 ; 17 ; 33 ; 32 ; 23 ) , premise selection ( 7 ; 1 ; 18 ; 27 ) , proof correction ( 17 ; 5 ) , and proof sketching ( 14 ; 25 ; 4 ; 5 ) . Optimizing these sub-tasks individually provides a modular path toward enhancing the global performance of the formalization pipeline. The open-source community currently lacks training data for tasks other than statement formalization and proof generation. Furthermore, the absence of expert models for the remaining sub-tasks necessitates the use of general-purpose large models as substitutes. Figure 1 : Schematic overview of the formalization workflow and its five sub-tasks. This paper focuses on data synthesis for these five sub-tasks and presents TheoremForge. This complete formalization workflow covers all the aforementioned tasks and serves as a data synthesis pipeline that achieves a favorable cost-performance trade-off. We utilize Gemini-3-Flash as the reasoning model to attain an average cost for a successful trajectory of $0.481. Existing formalization workflows aim primarily to improve success rates and incur extremely high test-time scaling costs ( 25 ; 4 ; 5 ) . These high costs manifest as large-scale parallel sampling and iterative subgoal decomposition which makes them unsuitable for large-scale data synthesis. The contributions of this paper are summarized below. • We design and open-source a formalization workflow capable of generating the full process from natural language mathematical propositions to formal proofs. • The workflow serves as a low-cost data synthesis pipeline for formal training data which covers multiple sub-tasks and facilitates the training of agentic models. • We evaluate the performance and cost of common formalization strategies under low inference budgets to provide a benchmark for future research. 2 Preliminaries and Notations We formally define the five sub-tasks and their data formats. For concrete data instances, please refer to Appendix C . Statement Formalization translates an informal statement S_{N} into a formal specification S_{F} conditioned on retrieved premises P . While syntax is compiler-verified, semantic equivalence is often evaluated via LLM-as-Judge 3 ; 8 to ensure the mathematical meaning is preserved. Training samples are triplets (S_{N},P,S_{F}) , where S_{F} is both syntactically and semantically valid. Proof Generation aims to generate a complete formal proof R_{F} from S_{F} and P that passes the kernel check. Unlike translation, the correctness of R_{F} is deterministically verifiable, providing an objective reasoning signal. Training samples are triplets (S_{F},P,R_{F}) . Premise Selection identifies pertinent definitions and theorems P from a library 35 using natural language queries Q based on S_{N}/S_{F} . Effective selection reduces context noise and prevents model overwhelm. Training samples are triplets (S_{N}/S_{F},Q,P) where premises directly contribute to a successful downstream formalization or proof. Proof Correction leverages compiler error messages E to repair a failed proof R_{F} into a successful version R_{F}^{\prime} 17 . This task emphasizes iterative refinement and precise fault localization, significantly enhancing efficiency compared to re-sampling 5 . Training samples are triplets (R_{F},E,R_{F}^{\prime}) . Proof Sketching decomposes complex theorems into manageable intermediate subgoals. Given S_{F} , P , and a natural language proof R_{N} , the model generates a compilable sketch R_{S} as a high-level reasoning roadmap 25 . Training samples are quadruplets (S_{F},P,R_{N},R_{S}) . Figure 2 : Statement Formalization Workflow. The pipeline consists of three phases: 1) Information Preprocessing for normalization and definition retrieval; 2) Expert Model Sampling for candidate generation and compilation checks; and 3) Statement Filtering for semantic verification via LLM-as-Judge and final selection. 3 System Design In this section, we present the architecture of TheoremForge. While the system functions as a robust solver through two primary stages defined as Statement Formalization and Proof Generation , its overarching design objective is to serve as a high-efficiency data synthesis pipeline. We explicitly architect the workflow to support a Decoupled Extraction Strategy which maximizes data yield by independently validating and harvesting intermediate outputs from every inference trajectory. We detail the execution logic of the formalization and proving stages followed by the specific mechanisms for data extraction. See Appendix A for details. 3.1 Statement Formalization The system must derive a syntactically correct and semantically aligned formal statement from the input prior to generating a proof. The workflow for this process is illustrated in Figure 2 . Information Preprocessing. Informal input statements exhibit diverse characteristics in both type and form. This heterogeneity often compels the formalization process to encompass ancillary tasks such as supplying implicit premises, solving for numerical answers, and interpreting ambiguous semantics. These factors can adversely affect model performance. We first employ a general-purpose LLM to denoise the input statement before engaging the expert formalizer to address these challenges. We subsequently implement an agentic premise selection strategy leveraging the open-source search engine LeanExplore 1 . The model generates search queries Q based on the normalized statement S_{N} and selectively incorporates valid premises P from the retrieval results