Just Type It in Isabelle! AI Agents Drafting, Mechanizing, and Generalizing from Human Hints
Kevin Kappelmann, Maximilian Schäffeler, Lukas Stevens, Mohammad Abdulaziz, Andrei Popescu, Dmitriy Traytel · 2026
Read Just Type It in Isabelle! AI Agents Drafting, Mechanizing, and Generalizing from Human Hints by Kevin Kappelmann, Maximilian Schäffeler, Lukas Stevens, Mohammad Abdulaziz, Andrei Popescu, Dmitriy Traytel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15713.txt.
Introduction
Kappelmann and five colleagues study full, lean type tags for rank-one polymorphic lambda terms as used in Isabelle, building on Smolka, Blanchette, and co-workers. They give a full formal spec with proofs, all built in Isabelle/HOL. A human and an LLM agent each draft pen-and-paper proofs on their own, and the agent turns both into Isabelle form, with human hints honing and widening the work.
Catalog record
| CREATOR | Kevin Kappelmann, Maximilian Schäffeler, Lukas Stevens, Mohammad Abdulaziz, Andrei Popescu, Dmitriy Traytel |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90586 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Just Type It in Isabelle! AI Agents Drafting, Mechanizing, and Generalizing from Human Hints Kevin Kappelmann, Maximilian Schäffeler, Lukas Stevens, Mohammad Abdulaziz, Andrei Popescu, Dmitriy Traytel Abstract Type annotations are essential when printing terms in a way that preserves their meaning under reparsing and type inference. We study the problem of complete and minimal type annotations for rank-one polymorphic λ \lambda -calculus terms, as used in Isabelle. Building on prior work by Smolka, Blanchette et al., we give a metatheoretical account of the problem, with a full formal specification and proofs, and formalize it in Isabelle/HOL. Our development is a series of experiments featuring human-driven and AI-driven formalization workflows: a human and an LLM-powered AI agent independently produce pen-and-paper proofs, and the AI agent autoformalizes both in Isabelle, with further human-hinted AI interventions refining and generalizing the development. 1 Introduction Some 30 years ago, Larry Paulson taught the working ML 1 1 1 Hereby, we refer to the programming language Standard ML, and its spiritual successors, implementations, and variants such as OCaml, Poly/ML and Isabelle/ML. programmers how to pretty print [ 34 ] . His textbook algorithm is mainly concerned with the printing layout for improved legibility, and has inspired the pretty printers of types, terms, and theorems used in the Isabelle proof assistant to this day. When printing Isabelle’s terms, or more generally terms of the typed lambda calculus ( Section 2 ), in addition to legibility, it is crucial to preserve the type information. Specifically, we are interested in the following round-trip property: starting from a fully typed term, we wish to print it in a way so that reparsing it and using type inference afterwards yields the original term. For example, printing the term (c_{\alpha},d_{\alpha}) 2 2 2 We prefer the compact t_{\tau} over Isabelle’s t::\tau type annotation syntax. with polymorphic constants c and d as (c,d) violates this property: invoking type inference yields the more general typing (c_{\alpha},d_{\beta}) . Achieving the round-trip hence requires type annotations. Since legibility remains a desideratum, these annotations should be as sparse as possible while still constraining type inference to recover the original term. Smolka, Blanchette et al. [ 39 , 38 , 8 ] identified these requirements and formulated the type annotation problem in the context of Isar proof reconstruction for Sledgehammer [ 33 ] . They call the round-trip property completeness , and they are interested in a complete solution that is also minimal . We use completeness and minimality as the correct printing problem ’s specification ( Section 3 ). Smolka, Blanchette et al. [ 39 , 38 , 8 ] also devised a solution to the correct printing problem, indicating that full minimality (w.r.t. a function assigning costs to annotation positions) is NP-hard, they settle for a tractable greedy algorithm, producing a complete, locally minimal solution. Their implementation is part of the Isabelle distribution and has been reused beyond Isar proof reconstruction in different contexts in which terms are printed: sketch and explore [ 20 ] , super_sketch [ 40 ] , AutoCorres2 [ 9 ] , types-to-sets [ 31 ] , and Apply2Isar [ 7 ] . Despite widespread usage, the algorithm’s implementation contained a completeness-compromising issue in one optimization. In the Isabelle2025-2 release, (c_{\alpha},d_{\alpha}) is printed as (c,d) —the type annotation is incorrectly dropped. In addition, it is difficult to convince Isabelle to actually print type annotations correctly when they are attached to constants and bound variables: []_{\mathsf{nat}\;\mathsf{list}} gets printed as [] and \lambda x.\,x_{\mathsf{nat}} as the garbled _type_constraint_ . We resolved these issues by correcting the optimization, forbidding annotations on bound variables in favor of annotating binders, and adding a custom print translation that preserves all annotations inserted by the algorithm. We also set up a testing environment for the algorithm using Mirabelle [ 17 ] . These implementation problems, however, also exposed a deeper issue: without a precise formal account of what the algorithm is supposed to guarantee, it is difficult to explain exactly why an optimization is unsound, to justify the repairs, or to rule out similar regressions. Alas, this paper is not about our technical refinements of the implementation. Our main goal as working programming-language metatheorists is to understand the type annotation problem precisely, with formal statements and complete proofs. (Smolka, Blanchette, et al. [ 39 , 38 , 8 ] give only informal claims and no proofs.) At the same time, we also identify as working metatheory formalizers and thus want a proof assistant to validate our proofs. Being modern working metatheorists and formalizers, we would like to conduct both activities assisted by modern technology, particularly large language models (LLMs), thus corroborating numerous recent accounts of their usefulness in our field. To this end, we conduct the following experiments contrasting human-driven and AI-driven activities, starting from Smolka, Blanchette, et al.’s informal description, the Isabelle/ML implementation, and an example algorithm execution: 1. A human expert formalizes the type annotation problem’s metatheory on paper. We present this in Sections 2 , 3 and 4 , an occasion to describe the problem in detail. 2. Independently, an LLM-powered AI agent formalizes the problem’s metatheory on paper. We discuss the experimental setup and how the result differs from the human pen-and-paper formalization ( Section 5 ). Drafts were iteratively refined through human reviews. Some feedback also went in the other direction: AI’s work has informed the human formalizer to prove local minimality. 3. A second AI agent (disconnected from the first) formalizes both pen-and-paper proofs in Isabelle/HOL. Both resulting autoformalizations 3 3 3 Autoformalization can refer to translating natural language statements to a formal system. Here, we adopt the deeper reading coined by Urban [ 42 ] , in which not only statements, but also proofs are translated to and checked by a proof assistant. use mostly the same setup; we discuss differences in the experience ( Section 6 ). 4. A human expert intervenes with the AI’s pen-and-paper formalization, observing that a core part of the algorithm could be generalized by reducing it to a standard problem, for which a rich theory as well as off-the-shelf algorithms exist. The AI agent analyses the human hint, shows the reduction to be correct, and updates the Isabelle formalization to do the same ( Section 7 ). To our own surprise, all experiments were successful. We have obtained three Isabelle formalizations of the desired results, without writing a single line of Isabelle ourselves. As our LLM, we used Claude Opus 4.6, a state-of-the-art general purpose model with strong reasoning capabilities—but, as far as we know, without a specific focus on interacting with proof assistants in general, nor Isabelle specifically. Claude proved a capable, although not logically infallible, companion: one that can substantially amplify the human experts’ work. In contrast to prior work in autoformalization ( Section 8 ), which typically starts from pen-and-paper proofs and/or targets classic mathematics, we begin with an algorithm that lacks formal correctness specifications. Another contribution is autoformalization of programming language metatheory, which has been largely unexplored. Finally, we are the first to report on a comprehensive autoformalization to Isabelle with a simple, reusable setup and best practices for the community. The input and output data of our experiments is available on