Final Technical Report, Comport: Rigorous Testing Methods to Safeguard Software Porting
Rubio Gonzalez, Cindy · 2026
Read Final Technical Report, Comport: Rigorous Testing Methods to Safeguard Software Porting by Rubio Gonzalez, Cindy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/doe-osti-3068739-final-technical-report-comport-rigorous.txt.
Introduction
Cindy Rubio-Gonzalez of the University of California, Davis, closes the ComPort project in this final report, dated April 28, 2026, under award SC0022182. The funding ran from September 2021 to August 2025. ComPort built testing methods to guard software when it is ported, moved from one computer type to another. Modern supercomputers mix many chip kinds, and that mix can quietly bend the numbers a trusted science code produces.
The team shipped five open-source tools and six papers, one winning the Hans Meuer Award for best paper at ISC High Performance 2023. CIGEN hunts input ranges that expose compiler-made number mismatches. FPLearner uses graph neural networks to tune mixed-precision programs, speeding prior tuners by about 25.5 percent on average. MUPPET applies mutation testing to squeeze out more speed.
Catalog record
| CREATOR | Rubio Gonzalez, Cindy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11 KB |
| LICENSE | Public Domain |
| RIGHTS | PUBLIC DOMAIN |
| SOURCE | DOE OSTI |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Final Technical Report ComPort: Rigorous Testing Methods to Safeguard Software Porting Cindy Rubio-González (PI) University of California, Davis April 28, 2026 1 Executive Summary This is the final technical report for the project “ComPort: Rigorous Testing Methods to Safe- guard Software Porting,” funded by the Department of Energy (DOE), Office of Science, Office of Advanced Scientific Computing Research (ASCR) under award number SC0022182. The report summarizes the work done over the funding period (9/2021-8/2025), focusing on the development of testing techniques to safeguard software porting. The research resulted in six publications in- cluding top-tier venues (ISC, ICSE, ICS, HPDC), the Hans Meuer Award at ISC High Performance 2023 for Best Paper, and the release of five open-source software tools. 2 Major Goals The primary objective of this project is to address the productivity and reliability crises in modern High Performance Computing (HPC) triggered by extreme hardware and software heterogeneity. While the integration of diverse accelerators and non-standard arithmetic enables unprecedented computational power, these advancements simultaneously compromise the numerical integrity of critical scientific applications. To solve this, the project aims to develop robust testing and de- bugging frameworks capable of identifying the subtle errors introduced by diverse CPUs, GPUs, and domain-specific accelerators. By establishing these methods, the project ensures that the shift toward complex, heterogeneous architectures does not undermine the accuracy or reliability of fundamental scientific results. 3 Technical Accomplishments This section details the project’s technical accomplishments, which includes a suite of methodologies and tools for triggering and isolating compiler-induced numerical inconsistencies, predicting the performance of mixed-precision floating-point programs, and applying performance optimizations via mutation testing. 3.1 Input Range Generation for Compiler-Induced Numerical Inconsistencies We developed CIGEN, an automated technique designed to identify input ranges that trigger compiler-induced numerical inconsistencies. CIGEN utilizes a multi-phase workflow that inte- grates input-partitioned and coverage-based sampling, input clustering, and optimization algo- rithms to uncover previously unknown problematic inputs. In an evaluation across 175 GNU 1 Scientific Library (GSL) functions, CIGEN demonstrated a 53.4% improvement over the state of the art in detecting inconsistencies. Furthermore, by integrating CIGEN with Ciel [1], we successfully identified 21 GSL functions where multiple, distinct sources of compiler-induced in- consistency were captured across different input ranges. This work was published at the In- ternational Conference on Supercomputing (ICS) 2024 [4] and the tool is publicly available at https://github.com/LLNL/CIGEN. 3.2 Expression Isolation in Heterogeneous Programs We developed Ciel, an automated approach for localizing numerical inconsistencies induced by compiler optimizations at the expression level in heterogeneous applications. Ciel utilizes a re- cursive bisection search algorithm with adaptive granularity to guide a floating-point precision enhancement strategy. This methodology allows for the efficient identification of specific pro- gram expressions responsible for numerical deviations. In experimental evaluations across diverse heterogeneous applications, Ciel achieved 99.4% precision in isolating inconsistencies and demon- strated performance comparable to or exceeding state-of-the-art methods for CPU-only programs. Ciel [1] was published at the International Supercomputing Conference (ISC) 2023, where it received the Hans Meuer Award (Best Research Paper). Ciel is open source and available at https://github.com/LLNL/Ciel. 3.3 Detection of Floating-Point Exceptions in AMD GPUs We developed FloatGuard, the first framework designed for the detection of floating-point ex- ceptions (such as NaNs and infinities) in HIP programs running on AMD GPUs. While similar tools existed for NVIDIA’s CUDA platform, AMD GPUs—now central to world-class exascale supercomputers like Frontier and El Capitan—have lacked a comparable tool. FloatGuard ad- dresses this gap by leveraging low-level AMD hardware registers and a novel iterative algorithm that combines assembly- and source-level code instrumentation with debugger-guided execution via ROCgdb. In an evaluation of 565 programs, the tool successfully detected exceptions in nearly 90% of cases with a performance slowdown that typically increases only linearly with the number of exceptions found. Furthermore, we demonstrated that aggressive compiler optimizations (e.g., -ffast-math) significantly altered the behavior and frequency of these exceptions, often reducing the total number reported by approximately 37%. FloatGuard was published at the International Sym- posium on High-Performance Parallel and Distributed Computing (HPDC’25) [5] and is available at https://github.com/LLNL/FloatGuard. 3.4 Predicting Performance and Accuracy for Precision Tuning We developed FPLearner, a deep learning-based framework that accelerates mixed-precision tuning by predicting the performance and accuracy of numerical programs. While traditional dynamic tuners suffer from scalability bottlenecks due to the requirement for exhaustive program executions, FPLearner replaces costly runs with rapid model predictions. By representing mixed- precision programs as graph structures, our approach utilizes Graph Neural Networks (GNNs) to model complex program dependencies. Experimental results show that FPLearner improves the efficiency of state-of-the-art dynamic tuners by an average of 25.5% (up to 61%) while maintaining or improving tuning quality. This work was published at the International Conference on Software Engineering (ICSE) 2024 [6]. The tool is open source and accessible at https://github.com/ ucd-plse/FPLearner. 2 3.5 Performance Optimization via Mutation Testing We developed MUPPET, an automated framework designed to optimize the performance of High Performance Computing (HPC) software by applying mutation testing at the source-code level. MUPPET dentifies potential performance improvements in OpenMP programs by injecting muta- tions—syntactic changes such as altering thread affinity or loop scheduling—and using exploratory algorithms like Bayesian Optimization to find configurations that increase speed without breaking correctness, successfully discovering speedup opportunities in over 75% of evaluated program, with the best speedup of 3.57x. MUPPET was first published at the International Workshop on Pro- gramming Models and Applications for Multicores and Manycores (PMAM’24) [2]. An extension of the work was published in the Journal of Parallel Computing (PARCO) [3]. MUPPET can be found at https://github.com/LLNL/MUPPET. 4 Professional Leadership and Community Engagement Beyond research dissemination and tool development, the PI demonstrated sustained leadership in the high-performance computing (HPC) community through the organization of major technical events. In collaboration with Lawrence Livermore National Laboratory (LLNL), during the review period the PI co-organized five editions (2021–2025) of the International Workshop on Software Correctness for HPC Applications (CORRECTNESS), held annually at the Supercomputing (SC) conference. The PI’s leadership extended to inter-agency coordination as a co-organizer of the DOE/NSF Workshop on Correctness in Scientific Computing at PLDI 2023. Furthermore, the PI prioritized educational outreach by co-developing and presenting tutorials at SC 2024 and SC 2025, as well as co-organizing the 2022 SIAM Mini-Symposium on Testing Numerical Code for Heterogeneity. These events were conducted in partnership with our collaborators at the University of Utah, the University of Washington, and Lawrence Livermore National Laboratory. 5 Concluding Remarks Over the duration of this award, the project successfully transformed the testing of numerical soft- ware for heterogeneous architectures. By developing a comprehensive ecosystem of tools, we have provided researchers with the means to trigger and isolate compiler-induced numerical inconsisten- cies often observed when porting code, as well as the means to optimize numerical applications. Furthermore, through our leadership of the CORRECTNESS workshop series, we have continued to foster a strong community. The open-source software and methodologies resulting from this work provide a lasting foundation for ensuring the reliability and efficiency of scientific applications. References [1] D. Miao, I. Laguna, and C. Rubio-González. Expression isolation of compiler-induced numer- ical inconsistencies in heterogeneous code. In A. Bhatele, J. R. Hammond, M. Baboulin, and C. Kruse, editors, High Performance Computing - 38th International Conference, ISC High Per- formance 2023, Hamburg, Germany,