Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learning
Ananyae Kumar Bhartari, Vinayak Vinayak, Vivek B Shenoy · 2025
Read Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learning by Ananyae Kumar Bhartari, Vinayak Vinayak, Vivek B Shenoy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11682.txt.
Introduction
Ananyae Kumar Bhartari, Vinayak Vinayak, and Vivek Shenoy wrote a paper from 2025 on a cheaper way to get high-order derivatives inside physics-guided learning. Such learning fits models to data while obeying a known equation, and it often needs second or fourth derivatives of the model's output. The standard tool takes them by repeated auto-differentiation, which is slow, memory-hungry, and shaky in noise.
Their module, named Mollifier Layers, swaps that tool for smoothing filters with known shapes. A derivative then becomes a weighted blend of nearby outputs, computed as a convolution. The module clips onto the output layer of any network with no other change. Across three network designs and equations up to fourth order, it cut memory and training time and sharpened parameter recovery. On real imaging data from cells, it read out reaction rates that vary across space.
Catalog record
| CREATOR | Ananyae Kumar Bhartari, Vinayak Vinayak, Vivek B Shenoy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11682
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learning Ananyae Kumar Bhartari, Vinayak Vinayak, Vivek B Shenoy Abstract Parameter estimation in inverse problems involving partial differential equations (PDEs) underpins modeling across scientific disciplines, especially when parameters vary in space or time. Physics-informed Machine Learning (PhiML) integrates PDE constraints into deep learning, but prevailing approaches depend on recursive automatic differentiation (autodiff), which produces inaccurate high-order derivatives, inflates memory usage, and underperforms in noisy settings. We propose Mollifier Layers, a lightweight, architecture-agnostic module that replaces autodiff with convolutional operations using analytically defined mollifiers. This reframing of derivative computation as smoothing integration enables efficient, noise-robust estimation of high-order derivatives directly from network outputs. Mollifier Layers attach at the output layer and require no architectural modifications. We compare them with three distinct architectures and benchmark performance across first-, second-, and fourth-order PDEs—including Langevin dynamics, heat diffusion, and reaction-diffusion systems—observing significant improvements in memory efficiency, training time and accuracy for parameter recovery across tasks. To demonstrate practical relevance, we apply Mollifier Layers to infer spatially varying epigenetic reaction rates from super-resolution chromatin imaging data—a real-world inverse problem with biomedical significance. Our results establish Mollifier Layers as an efficient and scalable tool for physics-constrained learning. 1 Introduction Parameter estimation through inverse problems constrained by PDEs arise across science and engineering, where parameters like diffusivity or conductance must be inferred from limited, noisy measurements Kaipio & Somersalo, 2006 ; Tarantola, 2005 ; Herrera et al., 2022 ; Zhang et al., 2020 . Traditional inference methods are computationally intensive, as they require repeatedly solving forward PDEs. These challenges grow worse in high-resolution or real-time settings, especially when governing parameters vary at fine scales or data is sparse Xun et al., 2013 . Deep learning has emerged as a powerful framework for such problems, with physics-informed machine learning (PhiML) approaches Toscano et al., 2024 —like PINNs Raissi et al., 2019 , DeepFNO Li et al., 2020 , and DeepONet Lu et al., 2021 —embedding PDE constraints into model training. These methods enable learning from limited observations, but rely heavily on recursive automatic differentiation (autodiff) to compute spatial and temporal derivatives Baydin et al., 2018 . Autodiff based PhiML techniques are memory-intensive and unstable for higher-order derivatives Wang et al., 2024 ; Fuks & Tchelepi, 2020 ; Fourment et al., 2023 ; Margossian, 2019 , making such models brittle under noise and costly for deep architectures. To overcome these limitations, we propose mollifier layers Friedrichs, 1944 ; Murio et al., 1998 : lightweight, architecture-agnostic modules that replace autodiff with convolution-based derivative estimation using smooth, compactly supported kernels. These layers estimate derivatives via convolution with analytic derivatives of smooth mollifier kernels, transferring differentiation off the network and implicitly regularizing the computation for stability and noise robustness. Integrated into and benchmarked against three PhiML architectures—PINNs Raissi et al., 2019 , PirateNet Wang et al., 2024 , and PINNsFormer Zhao et al., 2023 —mollifier layers offer a scalable, robust alternative to autodiff for physics-informed learning. They achieve significant improvements in parameter recovery, derivative stability, and computational efficiency across diverse PDEs, including a fourth-order reaction-diffusion model Heo et al., 2023 where they accurately infer the governing parameter while reducing the memory usage and time complexity by over an order of magnitude. Applied to super-resolution microscopy Schermelleh et al., 2019 of the human cell nuclei, they enable spatially resolved inference of critical biophysical parameters from noisy data, demonstrating practical utility in real-world scientific modeling. While our benchmarks target inverse PDEs, the underlying principle—offloading high-order derivative computation to analytic mollifier convolutions—can generalize to forward models Karniadakis et al., 2021 (shown briefly in B.5), operator learning Azizzadenesheli et al., 2024 , or neural ODE systems Ruiz-Balet & Zuazua, 2023 ; Chen et al., 2018 where gradient accuracy is crucial. Figure 1: Limitations of autodiff and overview of PhiML+Mollifier architecture. (a) Training time comparison for PINNs with and without PDE residual loss. (b) PINN-predicted vs. actual Laplacian in a reaction-diffusion system. (c) PINN-predicted vs. actual forcing term in a Langevin system. (d) PhiML+Mollifier Layer architecture replacing autodiff with mollifier-based convolution. 2 Motivation As shown in prior work Wang et al., 2024 ; Fuks & Tchelepi, 2020 ; Fourment et al., 2023 ; Margossian, 2019 , recursive autodiff—despite its central role in PhiML—poses key bottlenecks for learning PDEs, particularly in the accurate and efficient estimation of higher-order derivatives. To expose and quantify these, we benchmarked an upgraded PINN (details in A.1) on spatio-temporally varying parameter estimation. This analysis revealed three core issues—compute cost, derivative inaccuracy, and memory blow-up—that directly motivated our mollifier layer design. Motivation #1: Rising cost of high-order derivatives with model depth. Evaluating PDE residuals in PhiML requires multiple backward passes for each order of differentiation, causing compute time to grow superlinearly with network depth Cho et al., 2024 ; Suarez & Hecht, 2023 . This becomes especially burdensome in complex systems that demand deeper networks. For instance, in the reaction-diffusion benchmark (Sec. 4.3), training PINNs with 7 to 15 layers using the full loss (data + PDE) took significantly longer than using the data loss alone—an overhead that widens with depth (Fig. 1a). Such costs slow down training and escalate energy and hardware requirements, forming a major scalability barrier. Motivation #2: Inaccuracy in higher-order derivatives for high-frequency features. Recursive autodiff is susceptible to rounding errors from repeated gradient operations. This instability becomes particularly problematic when modeling spatio-temporally varying or noisy parameters. For example, a PINN trained to recover the gradient in a reaction-diffusion system achieved only 0.21 Laplacian correlation with ground truth (Fig. 1b), indicating poor quantitative accuracy. These issues are exacerbated with high-frequency or noisy signals, where the inherent smoothness of neural networks hinders the capture of sharp or discontinuous features Fuks & Tchelepi, 2020 . In a Langevin system with Gaussian white noise (Sec. 4.1.1), the PINN failed to capture variance in the inferred parameter (Fig. 1c). Together, these results underscore the fragility of autodiff in noisy, high-order settings, motivating the need for more stable and noise-resilient derivative estimation techniques. Table 1: Peak Memory usage by PINNs Equation MSE loss only MSE loss + PDE loss Langevin (first order 1D) 0.17 GB 0.21 GB Heat (second order 2D) 0.8 GB 1.2 GB Reaction-Diffusion (fourth order 2D) 0.4 GB 2.7 GB Motivation #3: High memory overhead from recursive autodiff. Computing higher-order derivatives through recursive autodiff requires storing intermediate gradients across layers, leading to memory usage that scales with both depth and derivative order. In experiments with parameter estimation for Langevin, heat, and reaction-diffusion equations using PINN(Sec. 4), adding the PDE error significantly increased GPU memory usage (hardware specs in A.6) compared to training with data-only losses (Table 1). This memory blow-up limits scalability and highlights the need for more memory-efficient alternatives. 3 Methods 3.1 Formulating parameter inference using PhiML To ground our innovations, we begin with a brief overview of parameter inference for PDEs using a general PhiML framework. Without loss of generality, consider a partial differential equation: u_{t}+D[u,\lambda]=0,x\in\Omega,t\in T (1) where u(t,x) is the solution of the PDE, D[.;\lambda] is a non-linear/linear ordinary or partial differential operator parameterized by \lambda(t,x) and the domain \Omega is a subset in R^{n} . Subscripts denote differentiation in time t or space x . Given this PDE,