ProofBuddy — How it Started, How it’s Going
Nadine Karsten, Kim Jana Eiken, Uwe Nestmann · 2025
Read ProofBuddy — How it Started, How it’s Going by Nadine Karsten, Kim Jana Eiken, Uwe Nestmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13474.txt.
Introduction
Nadine Karsten and co-authors report the journey of ProofBuddy, a web tool for teaching formal proofs with Isabelle. Students often struggle to learn proof craft from raw tools. A raw proof assistant can overwhelm beginners in class.
The team followed a design research path with classroom trials. Each trial's feedback reshaped the tool's next version. The web form beat desktop software on several fronts. Simplicity, upkeep, and tailoring all favored the browser. Interactive tutorials inside the tool drew special praise. Teachers could shape lessons to their own course needs. Students gained a gentler on-ramp to formal proof. The report shares lessons for education tool builders.
Catalog record
| CREATOR | Nadine Karsten, Kim Jana Eiken, Uwe Nestmann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13474
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ProofBuddy — How it Started, How it’s Going — Thanks: Our tool ProofBuddy should not be confused with another tool called “Proof Buddy” [ 9 ] . We first presented an extended abstract about ProofBuddy [ 17 ] in January 2023 at the TFPIE Workshop [ 23 ] , published as a more comprehensive description in the post-proceedings in August 2023 [ 16 ] . “Proof Buddy” is a tool by Steve Earth, Jeremy Johnson and Bruce Char, presented at a demo session in March 2023 at SIGCSE 2023 [ 9 ] . Affiliation: Technische Universität Berlin, Germany Nadine Karsten, Kim Jana Eiken, Uwe Nestmann Abstract We report on our journey to develop ProofBuddy , a web application that is powered by a server-side instance of the proof assistant Isabelle, for the teaching and learning of proofs and proving. The journey started from an attempt to use just Isabelle in an educational context. Along the way, following the educational design research approach with a series of experiments and their evaluations, we observed that a web application like ProofBuddy has many advantages over a desktop application, for developers and teachers as well as for students. In summary, the advantages cover simplicity, maintainability and customizability. We particularly highlight the latter by exhibiting the potential of interactive tutorials and their implementation within ProofBuddy . 1 Introduction Motivation. Students in Computer Science (CS) often encounter difficulties in developing and writing proofs by themselves, as highlighted, e.g., by Frede and Knobelsdorf [ 11 , 19 ] . The primary challenges lie in the use of “formal” and “mathematical” language, but also in assessing the correctness and completeness of proofs [ 18 ] . As CS students are used to interact with computers for acquiring competences in programming, among other activities, it seems natural to provide CS students also with computer-based tools for the acquisition of competences in proving. Various strategies have been proposed along these lines aiming to provide students with much more immediate and personalized feedback on their own learning process. In the remainder of this paper, we avoid the use of the terms “formal” and “mathematical” language, as they can be misleading. Instead, we stick to the following convention for three levels of language with varying degrees of “formality” [ 5 ] : (i) We use natural language to refer to what one can find in typical textbooks, i.e., a “pseudo-natural” language, according to socio-mathematical norms [ 36 ] , sprinkled with mathematical or logical formulae. (ii) We use logical language to refer to a formal representation of proofs within some logical system, e.g., using natural deduction [ 13 , 10 ] . (iii) We use tool [input] language to refer to the input language of a tool ( Section 2 ) that supports the construction of proofs. The “teaching and learning of proofs and proving” (TLPP) is of course not a topic originating only from CS, but has a long tradition in the field of Mathematics education. There, the notion of proof competence has been developed and refined over many years. In order to use this notion to classify and evaluate the possible effects of computer-based tools for TLPP, we explain it in more detail. Proof competence typically includes four dimensions of competence [ 8 , 21 ] : (1) Professional competence comprises the knowledge about the topic of the proof task. (2) Representation competence describes the ability to use a “sufficiently formal” language to write a proof. (3) Communication competence characterizes the ability to discuss the finding of a proof and the solution. (4) Methodological competence comprises three parts [ 14 , 8 ] : (i) the proof scheme involves the use of deductive patterns for proving assertions, (ii) the proof structure entails the systematic organization of a proof and (iii) the chain of conclusions refers to the sequential arrangement of steps within a proof, forming a coherent chain of arguments. ProofBuddy . In [ 16 ] , we introduced our own computer-based tool to support TLPP: ProofBuddy 1 1 1 proofbuddy.tu-berlin.de is a web application that is based on the interactive theorem prover Isabelle, having it running in the backend. Opting for a web application format as opposed to the use of Isabelle/jEdit [ 39 ] (the IDE that is part of the Isabelle software package) has the advantage of enabling the collection of a rich dataset for analyzing learning behaviors and outcomes, next to the unneeded local installation of Isabelle [ 16 ] . As also announced in [ 16 ] , we carried out teaching experiments via a dedicated TLPP-course at Bachelor level to improve and evaluate our tool and our TLPP-materials. We decided to follow the educational design research approach (see next paragraph) to systematically and consciously drive the development process of ProofBuddy from v0 up to v2 . Educational design research. Educational design research (EDR) [ 27 , 25 , 32 ] refers to a comprehensive approach in which educational theories are applied to concrete educational challenges, for example, to improve a university course to better achieve its goals. In this context, researchers and teachers often work closely together to try out creative ideas and to test them continuously and iteratively. First, the given educational challenge must be carefully analyzed to work out suggestions for specific interventions. Then, researchers design systems, often using technologies, to devise teaching and learning materials and methods to achieve the predicted learning gains based on some educational theory. The response of teachers and students to specific design features suggested by the theory is then examined by the designers as the systems are put into practice and analyzed. The researchers then start the next iteration to either try out another idea for the original problem or to identify the next biggest problem. Given the inherent uncertainties of this process, numerous smaller and less controlled studies about EDR have been conducted. In a given concrete EDR scenario, it is important to focus on one problem at a time, to understand it, to determine the interventions that might lead to success, to build prototypes and to test them. As a matter of fact, EDR carries the risk of not necessarily leading to clearly identifiable or measurable success. While the first iteration often leads to weak results, the availability of various design options facilitates the refinement and enhancement of outcomes through direct comparisons across iterations. The results of such studies are meaningful even if there is no control group (which would be useful to compare the outcome of an experiment with and without a tested intervention), because the changes between the iterations are usually small. Contribution. In this paper, we report on our journey to iteratively run teaching experiments following the EDR approach and share our learnings along the way. We developed ProofBuddy over three iterations, starting with the prototype ProofBuddy v0 , as presented in [ 16 ] , in order to facilitate TLPP. To better guide students on their learning path, we introduce interactive tutorials for ProofBuddy v1 . To the best of our knowledge, the iterative and joint development of a TLPP-tool, TLPP-materials and an associated course structure, together with systematic evaluations that are rooted in didactics research and that are based