The Neural Compiler: Program-to-Network Translation for Hybrid Scientific Machine Learning

Lucas Sheneman · 2026

Read The Neural Compiler: Program-to-Network Translation for Hybrid Scientific Machine Learning by Lucas Sheneman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22498.txt.

Introduction

Sheneman presents the Neural Compiler, translating Scheme-like programs into frozen, differentiable PyTorch modules. Compiled modules match the source program to floating-point precision and provide gradients. In hybrid models, compiled modules encode known physics exactly. Across six domains, compiled modules match hand-coded implementations numerically. With only 1 to 4 trainable parameters, compiled models recover physical constants to under 1 percent error in most cases. PINN baselines with over 8500 parameters show 7 to 93 percent error. Compiled modules also compose with zero error. The main value is composability.

Catalog record

CREATORLucas Sheneman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55390
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22498

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Neural Compiler: Program-to-Network Translation for Hybrid Scientific Machine Learning Lucas Sheneman Abstract Scientific machine learning frequently requires integrating known physics with unknown components: the equation form is known, but parameters or correction terms must be learned from data. Current approaches either discard the known structure entirely (neural networks), encode it as a soft penalty that the model may violate (physics-informed neural networks), or require hand-implementation for each new equation (manual PyTorch coding). We present The Neural Compiler , a system that translates programs in a first-order expression language with Scheme syntax into frozen, differentiable PyTorch modules. These compiled modules compute exactly : they produce the same output as the source program to floating-point precision, with exact gradients via autograd. The compiled component contributes zero approximation error on its safe domain, including outside the training distribution; in hybrid architectures, approximation error arises only from the learned components, while the compiled module encodes known physics exactly. We evaluate the compiler across six experiments spanning algebraic equations (15 Feynman physics laws), ordinary differential equations (Lotka-Volterra, damped pendulum), a partial differential equation (1D heat equation), compositional generalization, and 3D vector mechanics. Against four baseline types (hand-coded PyTorch (same equations without the compiler), physics-informed neural networks (PINNs), neural ODEs, and pure MLPs) we find that: (1) compiled and hand-coded models produce numerically identical results for single equations, confirming zero numerical discrepancy relative to hand-coded PyTorch; (2) compiled models recover physical constants to < < 1% error with 1–4 trainable parameters where PINNs with 8,500+ parameters show 7–93% error; (3) compiled modules compose with zero error at arbitrary depth while neural approximations accumulate errors up to 5.9 × 10 9 5.9\times 10^{9} in high-depth chains; and (4) the compiler’s value is systematic composability : generating correct, differentiable modules from symbolic specifications rather than hand-coding each equation. The system supports 51 primitive operations including vector and matrix algebra, enabling PDE discretizations. We discuss how large language models can serve as a natural-language front end, translating physics descriptions into compilable programs and opening a path toward self-architecting scientific models. 1 Introduction A central tension in scientific machine learning is the gap between what we know and what we can encode . Researchers frequently possess partial domain knowledge: the gravitational force follows an inverse-square law, but the drag coefficient is unknown; the heat equation governs diffusion, but the thermal diffusivity must be learned from data; the predator-prey dynamics follow Lotka-Volterra equations, but the rate constants are uncertain. In each case, the scientist holds a program , a symbolic expression describing the known physics, and needs to integrate it with a trainable model that learns the rest. This integration is surprisingly difficult. The dominant paradigm in scientific ML offers three unsatisfying options: Option 1: Discard the program. Train a neural network from scratch on input-output data. Neural ODEs [ 5 ] learn the entire ODE right-hand side; standard MLPs learn arbitrary function mappings. This approach is general but wasteful: it must rediscover structure that the scientist already knows. A 3-layer MLP with 8,700 parameters learning the pendulum equation \ddot{\theta}=-(g/L)\sin\theta-b\dot{\theta} must approximate both the \sin function and the parametric dependence from data alone. It uses 4,000 \times more parameters than needed, fails on extrapolation, and its learned weights are uninterpretable. Option 2: Encode physics as a soft penalty. Physics-informed neural networks (PINNs) [ 16 ] add the governing equation as a regularization loss: the network is penalized for violating the physics but free to do so if the data loss decreases. This “soft constraint” approach is elegant in principle but fragile in practice. The balance between data and physics losses requires careful tuning. More fundamentally, soft constraints provide no guarantee: the model can, and often does, violate the physics, especially when extrapolating. In our experiments, a PINN trained on heat equation data recovers the thermal diffusivity \alpha with 93% error, while a hard-constrained compiled model achieves 0.0% error (Section 3.4 ). Option 3: Hand-code each equation. Write the physics directly as a PyTorch nn.Module with trainable nn.Parameter entries. This produces exact computation with exact gradients, but it requires manual reimplementation for every new equation, is error-prone for complex expressions, and, critically, does not compose. A library of 15 hand-coded Feynman equations cannot be automatically chained, recombined, or extended without writing new code for each combination. The compiler alternative. We introduce a fourth option: compile the known physics from a symbolic specification into a differentiable module, automatically. Given a program in a first-order expression language with Scheme syntax (e.g., (* (- 0 (/ g_L 1)) (sin theta)) ) for pendulum gravity, the Neural Compiler produces a frozen PyTorch nn.Module that: 1. Computes exactly. The module produces the same output as the source program to floating-point precision for any input in the safe domain (Theorem 2 ). 2. Provides exact gradients. Autograd through the compiled module yields the true derivative of the program, enabling gradient-based learning of exposed parameters (Theorem 3 ). 3. Exact on the safe domain. The compiled component contributes zero approximation error at any input in its safe domain, including outside the training distribution; all approximation error comes from the learned components (Theorem 5 ). 4. Composes exactly. Chains of compiled modules maintain zero error at arbitrary depth, while neural approximation errors can be amplified multiplicatively by downstream Lipschitz constants (Proposition 6 ). For a single equation, the compiler produces output identical to a hand-coded PyTorch implementation; our experiments confirm this to machine precision across all six experiment domains (Tables 1 – 4 ). The compiler’s value is not numerical superiority over hand-coding, but systematic composability : it provides a uniform mechanism to translate any symbolic expression into a correct, differentiable, composable module. A library of compiled equations can be chained, recombined, and extended by changing text strings rather than writing new code. This is the foundation for programmatic physics-module generation, including by large language models (Section 7 ). Contributions. 1. A compiler from first-order arithmetic expressions (with Scheme syntax) to frozen, differentiable PyTorch modules, supporting 51 primitive operations including vector and matrix algebra. 2. Formal guarantees of compilation correctness, gradient exactness, compiled-component exactness, and composition error bounds. 3. Hybrid architecture patterns (known structure with unknown parameters, known terms plus learned corrections, and compositional pipelines) demonstrated on six experiment domains. 4. Systematic comparison against four baseline types (hand-coded PyTorch, PINNs, neural ODEs, pure MLPs) establishing that hard-constraint compilation provides substantially better parameter recovery and extrapolation than soft-constraint approaches, especially for transcendental, PDE, and compositional settings. Scope. For any single equation, the compiled module produces output numerically identical to a hand-coded PyTorch implementation. If a practitioner has one equation and is willing to implement it manually, the compiler offers no accuracy benefit. The compiler’s value is systematic generation, composability, and the ability to serve as a target for LLM-driven scientific model construction (Section 7 ). 2 The Neural Compiler 2.1 Source Language The source language \mathcal{L} is a first-order, pure arithmetic expression language with Scheme syntax; programs are total on their safe domain (Definition 1 ), where partial operations (division, log, sqrt) are well-defined. Programs are built from constants c\in\mathbb{R} , input variables, let-bindings, conditionals, and 51 primitive operations organized in four categories: • Scalar arithmetic (24 ops): +,-,\times,\div ,

Read the complete text