Do Unit Proofs Work? An Empirical Study of Compositional Bounded Model Checking for Memory Safety Verification Conference: 48th IEEE/ACM International Conference on Software Engineering; 2026; Rio De Janeiro, Brazil DOI: DOI ISBN: ISBN

Paschal C. Amusuo, Owen Cochell, Taylor Le Lievre, Parth V. Patil, Aravind Machiry, James C. Davis · 2025

Read Do Unit Proofs Work? An Empirical Study of Compositional Bounded Model Checking for Memory Safety Verification Conference: 48th IEEE/ACM International Conference on Software Engineering; 2026; Rio De Janeiro, Brazil DOI: DOI ISBN: ISBN by Paschal C. Amusuo, Owen Cochell, Taylor Le Lievre, Parth V. Patil, Aravind Machiry, James C. Davis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13762.txt.

Introduction

Paschal Amusuo and co-authors run the first systematic study of unit proofs, a formal method for checking memory safety in software. Their 2025 paper deals with bounded model checking done piece by piece. Each unit of code gets a proof harness. Practice varies widely between and within teams, causing errors and missed bugs. No standard method or evidence base existed.

The team built a method that uses checker feedback and clear criteria to write proofs. They wrote seventy three proofs across four embedded operating systems. The proofs caught seventy four percent of recreated defects. Raising the check depth caught nine percent more. Nineteen new defects surfaced as well. Proofs were small: about eighty seven minutes to write and sixty one to run, on average. The authors give engineers practical guidance from the data.

Catalog record

CREATORPaschal C. Amusuo, Owen Cochell, Taylor Le Lievre, Parth V. Patil, Aravind Machiry, James C. Davis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13762

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do Unit Proofs Work? An Empirical Study of Compositional Bounded Model Checking for Memory Safety Verification Conference: 48th IEEE/ACM International Conference on Software Engineering; 2026; Rio De Janeiro, Brazil DOI: DOI ISBN: ISBN Paschal C. Amusuo, Owen Cochell, Taylor Le Lievre, Parth V. Patil, Aravind Machiry, James C. Davis Abstract . Memory safety defects pose a major threat to software reliability, enabling cyberattacks, outages, and crashes. To mitigate these risks, organizations adopt Compositional Bounded Model Checking (BMC), using unit proofs to formally verify memory safety. However, methods for creating unit proofs vary across organizations and are inconsistent within the same project, leading to errors and missed defects. In addition, unit proofing remains understudied, with no systematic development methods or empirical evaluations. This work presents the first empirical study on unit proofing for memory safety verification. We introduce a systematic method for creating unit proofs that leverages verification feedback and objective criteria. Using this approach, we develop 73 unit proofs for four embedded operating systems and evaluate their effectiveness, characteristics, cost, and generalizability. Our results show unit proofs are cost-effective, detecting 74% of recreated defects, with an additional 9% found with increased BMC bounds, and 19 new defects exposed. We also found that embedded software requires small unit proofs, which can be developed in 87 minutes and executed in 61 seconds on average. These findings provide practical guidance for engineers and empirical data to inform tooling design. 1. Introduction Memory safety defects ( Aleph One 2025 ) remain a significant threat to the reliability of critical software systems, particularly in programs written in C and C++. These defects facilitate cybersecurity attacks ( noa 2025i ) , software crashes ( noa 2025f ) and outages ( noa 2024a ) . A notable example is the July 2024 CrowdStrike outage ( noa 2024a ) , caused by an out-of-bounds read ( noa 2025b ) , affecting 8.5 million systems ( Weston 2025 ) and resulting in a $10 billion loss ( Wee 2025 ) . Additionally, these defects account for up to 70% of high-severity vulnerabilities ( Alex Rebert et al. 2025b ; MSRC Team 2025 ) in C/C++ codebases. While memory-safe languages like Rust exist, many active C/C++ projects remain, with barriers to transition, and many Rust projects still depend on C/C++ libraries and unsafe code ( Sharma et al. 2025 ; Fulton et al. 2025 ) . Given the severity of these issues, government agencies ( Bob Lord 2025 ; noa 2025d ) , academia ( Watson et al. 2025 ) , and industry ( Alex Rebert et al. 2025a ) increasingly advocate for memory-safe development practices and stricter safety standards. Particularly, the United States Department of Defense encourages the use of formal methods to mitigate these critical defects ( noa 2025d ) . Figure 1. A unit proof, containing models for func1 and func4, verifies the functional unit comprising func2 and func3. To mitigate memory safety defects, AWS ( Chong et al. 2020 ) , ARM ( Wu et al. 2024 ) , and Ant Group ( Wang et al. 2025b ) employ Bounded Model Checking (BMC) to formally verify memory safety of software systems. They adopt a compositional approach, known as unit proofing ( Amusuo et al. 2025 ) , to verify functional units individually. This approach relies on unit proofs that correctly model a unit’s environment for verification. However, there is no standardized method for creating unit proofs to ensure correctness. Different organizations employ varying methodologies, leading to inconsistencies even within the same projects. The lack of a systematic approach introduces errors in unit proofs ( section 5.4 ), prevents the detection of defects ( Amusuo et al. 2025 ) , and increases the cost of adoption for new organizations. In addition, unit proofing remains understudied in research. Existing experience reports ( Chong et al. 2020 ; Wu et al. 2024 ; Wang et al. 2025b ) only provide high-level descriptions without empirical validation. Prior studies on formal verification ( Matichuk et al. 2015 ; Olszewska and Sere 2010 ; Beyer and Lemberger 2017 ) do not address unit proofing or memory safety verification. As a result, no systematic methods exist for developing unit proofs and there are no empirical evaluations of unit proofing’s effectiveness or costs. This work presents the first empirical study on unit proofing for memory safety verification and addresses five research questions. We evaluate on embedded software due to its safety-critical applications ( Feiler 2025 ) and the challenges of adopting alternative techniques ( section 4.1.1 ). First, we introduce a systematic method for unit proof creation, where coverage and error reports guide the unit proof development, and objective criteria are used to assess completion. Using this method, we created 73 unit proofs to verify functional units in four embedded operating systems. Using a blind approach ( section 4.3.1 ), we then reintroduce 89 known memory safety defects in these units and evaluate the effectiveness of unit proofs in detecting the recreated defects, analyze the unit proof characteristics and measure the cost of developing and using them. Finally, we compare them with expert-developed unit proofs and assess if the systematic approach will generalize to unverified functions. Our results provide empirical measurements of the benefits and costs of using unit proofs for memory safety verification. Of 89 recreated defects, systematic unit proofs detected 66 (74%) and an additional 8 (9%) with increased BMC bounds, while 10 remained undetected due to memory exhaustion. Additionally, unit proofs exposed 19 new defects, highlighting their practical impact. On cost, developing each unit proof took an average of 87 minutes, with execution averaging 61 seconds. The median unit proof was developed in 72 minutes, executed in 25 seconds, and verified 185 lines of C code. Our results also show that unit proof sizes correlate with functional unit size ( R^{2}=0.328 ); that verification formula size better predicts execution time than does program size ( R^{2}=0.852 vs. R^{2}=0.029 ); and that systematic unit proofs are smaller, faster, and achieve better coverage than those developed by project experts. In summary, our main contributions are: • A systematic approach to developing unit proofs for memory safety, including strategies for deriving unit proof models and loop bounds. • The first empirical study on the use of unit proofs for memory safety verification. Our results provide evidence that unit proofs are cost-effective for ensuring memory safety. • A dataset containing 73 unit proofs, 89 recreated and 19 new memory safety defects to facilitate research on unit proofing. Significance : As software has become more critical to society, formal methods are now seeing broader industry adoption. Our investigation aims for two benefits: (1) to help engineering teams to develop effective unit proofs, and (2) to guide research on automating unit proof development, thereby reducing unit proofing costs and enabling broader adoption of memory safety verification in software engineering practices. 2. Background & Related Work In this section, we cover the memory safety concept and how it can be verified using Bounded Model Checking (BMC). Then we discuss approaches for applying BMC to larger software and review the empirical studies on formal verification. 2.1. Memory Safety The C and C++ languages allow software engineers to manipulate memory

Read the complete text