Numerical Superoptimization for Library Learning
Jonas Regehr, Mitch Briles, Zachary Tatlock, Pavel Panchekha · 2026
Read Numerical Superoptimization for Library Learning by Jonas Regehr, Mitch Briles, Zachary Tatlock, Pavel Panchekha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24812.txt.
Introduction
Regehr and co-authors turn superoptimisation toward library learning. Fast math libraries need expert implementations of primitives like sine and exponential, but which new ones are worth building is open. Superoptimizers already enumerate candidate primitives during search, and can price their counterfactual value. Their system, GrowLibm, repurposes the Herbie superoptimizer. It mines candidates from search results and ranks them by estimated workload gain. Across three applications, it finds compact primitive sets with strong speed benefits.
Catalog record
| CREATOR | Jonas Regehr, Mitch Briles, Zachary Tatlock, Pavel Panchekha |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77621 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.24812
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Numerical Superoptimization for Library Learning Jonas Regehr, Mitch Briles, Zachary Tatlock, Pavel Panchekha Abstract . Numerical software depends on fast, accurate implementations of mathematical primitives like sin \sin , exp \exp , and log \log . Modern superoptimizers can optimize floating-point kernels against a given set of such primitives, but a more fundamental question remains open: which new primitives are worth implementing in the first place? We formulate this as numerical library learning : given a workload of floating-point kernels, identify the mathematical primitives whose expert implementations would most improve speed and accuracy. Our key insight is that numerical superoptimizers already have the machinery well-suited to this problem. Their search procedures happen to enumerate candidate primitives, their equivalence procedures can generalize and deduplicate candidates, and their cost models can estimate counterfactual utility : how much the workload would improve if a given primitive were available. We present GrowLibm , which repurposes the Herbie superoptimizer as a numerical library learner. GrowLibm mines candidate primitives from the superoptimizer’s intermediate search results, ranks them by counterfactual utility, and prunes redundant candidates. Across three scientific applications (PROJ, CoolProp, and Basilisk), GrowLibm identifies compact, reusable primitives that can be implemented effectively using standard numerical techniques. When Herbie is extended with these expert implementations, kernel speed improves by up to 2.2 × 2.2\times at fixed accuracy, and maximum achievable accuracy also improves, in one case from 56.0 % 56.0\% to 93.5 % 93.5\% . We also prototype an LLVM matcher that recognizes learned primitives in optimized IR, recovering 26 replacement sites across five PROJ projections and demonstrating measurable effects on both end-to-end speed and accuracy. 1. Introduction Maps and navigation, physical simulation, engineering CAD—computation in these increasingly important fields relies on short floating-point kernels whose speed and accuracy shape application behavior. Those kernels, in turn, are built from the mathematical primitives available on the target platform. These primitives include basic arithmetic operators as well as elementary functions provided by system math libraries like \sin , \exp , and \log . They might also include third-party libraries like GSL, or even application-specific library functions written by experts. Floating-point kernels then compose these primitives into higher-level mathematical operations while taking care to ensure both speed and accuracy. In fact, the programming languages community has spent over a decade developing tooling to support this task. Modern numerical superoptimizers like Herbie ( Panchekha et al. 2015 ) , for example, can increasingly optimize kernels against a given set of available primitives ( Saiki et al. 2025 ) , even scaling the results to real-world applications like the LULESH hydrodynamics application ( Qian et al. 2026 ) . Modern verification tools like Satire ( Das et al. 2020 ) make it possible to statically verify these implementations, given error models for those primitives. And modern precision tuners like FPTuner ( Chiang et al. 2017 ) and POP ( Ben Khalifa et al. 2020 ) can even optimize kernels to leverage mixed precision. Designing a particular floating-point kernel against a particular set of primitives is increasingly well supported. This only raises the question: which new primitives are worth building? Some primitives, like \sin or matrix multiplication, are so widely used that implementation justifies itself. But how should library authors prioritize the zoo of mathematical functions found across fields as diverse as geospatial projections, Helmholtz energy formulations, astrodynamics, and more? How do library authors determine which primitives are most common, and most important, to their particular domain? We call this problem numerical library learning : given an application and a representative workload, identify mathematical primitives whose specialized implementations would most improve the application’s speed/accuracy tradeoff. Numerics are an especially attractive setting because primitives are small, pure expressions, while their implementations can range widely, covering algebraic identities, multi-word arithmetic, approximations of various types, digit recurrences, and even hardware-aware tuning. In other words, primitives are easy to characterize but require expertise to implement well. Numerical library learning, then, prioritizes that expertise. Our key insight is that numerical superoptimizers already have the machinery of library learners. Their search procedures enumerate plausible primitives, their equivalence procedures identify different presentations of the same latent function, and their error and cost models answer the question library designers actually care about: if this primitive existed, where would it be used, and how much would it help? We present GrowLibm , a numerical library learner that uses a numerical superoptimizer “in reverse”: instead of optimizing a kernel against a fixed set of primitives, it asks which new primitives would improve many kernels at once. GrowLibm mines candidate primitives from the superoptimizer’s own search space, semantically deduplicates and generalizes them, and ranks them by counterfactual utility : how much the workload would improve if the primitive were available. Intuitively, GrowLibm favors candidates that are frequent, small, yet still hard for a superoptimizer to optimize on its own. The result is a short list at the right scope for expert attention: compact, reusable primitives that recur across kernels and unlock better implementations. We evaluate GrowLibm on three substantial scientific applications: cartographic projections in PROJ ( PROJ contributors 2025 ) , thermophysical property computations in CoolProp ( Bell et al. 2014 ) , and astrodynamics kernels in Basilisk ( Kenneally et al. 2020 ) . Across these applications, GrowLibm proposes compact domain-specific primitives that a numerical expert can implement effectively. When implemented well, many of these primitives are reused in optimized kernels and expand the speed/accuracy frontier, with speed-ups of up to 2.2\times compared to similarly-optimized kernels without access to GrowLibm -suggested primitives. The new primitives also enable higher accuracy than previously possible, with one kernel in PROJ going from 56.0\% to 93.5\% accurate thanks to its use of 2 GrowLibm -suggested primitives. We also prototype an LLVM matcher that recognizes learned primitives in optimized LLVM IR, recovering 26 replacement sites across five PROJ projections. Some projections are both faster (by up to 5\% ) and more accurate; others trade modest slowdowns for substantially improved accuracy, illustrating how learned primitives expand the deployment-level speed/accuracy frontier. Contributions. • We formulate numerical library learning , the workload-driven problem of deciding which new mathematical primitives are worth expert implementation for a given numerical application. • We present GrowLibm , which repurposes the Herbie superoptimizer as a library learner by mining candidates from its search space, semantically deduplicating and generalizing them, and ranking them by counterfactual utility. • We show that, across PROJ, CoolProp, and Basilisk, GrowLibm finds compact primitives that a numerical expert can implement effectively, that Herbie can reuse in optimized kernels, and that expand the reachable speed/accuracy frontier. • We demonstrate a path to deployment with a prototype LLVM matcher that recognizes learned primitives in optimized IR and rewrites them to deployed implementations. Herbie Workload: source kernels Platform: default Extract subs & cut at operators Canonicalize & deduplicate Generation Convert to FPCore Numerical kernels Herbie Workload: candidates Platform: grow Rank by frequency, urgency, and size Add top candidates to platform (deduplicate by implication) Selection Herbie Workload: source kernels Platform: grow Filter by number of uses Proposed primitives Figure 1. The GrowLibm pipeline. Given numerical kernels converted to FPCore ( FPBench Project 2021 ; Damouche et al. 2016 ) , a standard floating-point interchange format, the