Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss

Filippo Quarenghi, Ryan Cotsakis, Tom Beucler · 2026

Read Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss by Filippo Quarenghi, Ryan Cotsakis, Tom Beucler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11422.txt.

Introduction

Quarenghi, Cotsakis, and Beucler bridge the training gap in Earth deep learning. Models cannot train directly on non-smooth metrics, so smooth proxies blur fine detail. They offer two routes. One relaxes discrete topological operations with softened thresholds and continuous logic. The other learns smooth stand-ins with Lipschitz networks, steadied by spectral normalization and geometric rules. Applied to the Minkowski image loss for rainfall fields, the constrained stand-in removes geometric violations seen in plain baselines. Strict Lipschitz rules, though, over-smooth gradients and limit recovery of fine textures.

Catalog record

CREATORFilippo Quarenghi, Ryan Cotsakis, Tom Beucler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60323
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11422

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss Filippo Quarenghi, Ryan Cotsakis, Tom Beucler Abstract The “differentiability gap” presents a primary bottleneck in Earth system deep learning: since models cannot be trained directly on non-differentiable scientific metrics and must rely on smooth proxies (e.g., MSE), they often fail to capture high-frequency details, yielding “blurry” outputs. We develop a framework that bridges this gap using two different methods to deal with non-differentiable functions: the first is to analytically approximate the original non-differentiable function into a differentiable equivalent one; the second is to learn differentiable surrogates for scientific functionals. We formulate the analytical approximation by relaxing discrete topological operations using temperature-controlled sigmoids and continuous logical operators. Conversely, our neural emulator uses Lipschitz-convolutional neural networks to stabilize gradient learning via: (1) spectral normalization to bound the Lipschitz constant; and (2) hard architectural constraints enforcing geometric principles. We demonstrate this framework’s utility by developing the Minkowski image loss, a differentiable equivalent for the integral-geometric measures of surface precipitation fields (area, perimeter, connected components). Validated on the EUMETNET OPERA dataset, our constrained neural surrogate achieves high emulation accuracy, completely eliminating the geometric violations observed in unconstrained baselines, which generate physically impossible precipitation fields in up to 7.8% of cases. However, applying these differentiable surrogates to a deterministic super-resolution task reveals a fundamental trade-off: while strict Lipschitz regularization ensures optimization stability, it inherently over-smooths gradient signals, restricting the recovery of highly localized convective textures. This work highlights the necessity of coupling such topological constraints with stochastic generative architectures to achieve full morphological realism. K eywords differentiability \cdot knowledge-guided machine learning \cdot Minkowski functionals \cdot radar precipitation \cdot super-resolution 1 Introduction Structural realism in predicted precipitation fields depends on metrics that are inherently non-differentiable, creating a bottleneck for gradient-based optimization. To bridge this gap, we leverage knowledge from integral geometry, which reveals organizational biases often missed by standard statistical verification ( 1 ) . Unlike recent work exploring surrogates in latent spaces ( 17 ) , we enforce these properties directly in the optimization landscape. We introduce a framework situated at the intersection of differentiable programming ( 6 ) and knowledge-guided machine learning ( 13 ) . Within this framework, we compare an analytical approximation against a neural emulator, both designed to map the level sets of precipitation fields to key integral-geometric measures (area, perimeter, and number of connected components). By relaxing this discrete combinatorial problem into a continuous regression task ( 11 ) , we enable the direct backpropagation of structural error signals. Crucially, we ensure emulation stability adopting a knowledge-guided Lipschitz-convolutional neural network (Lip-CNN) ( 7 ) . This architecture combines spectral normalization—to prevent exploding gradients—with hard geometric constraints (monotonicity, isoperimetry) embedded in the computation graph. Concurrent efforts to integrate non-differentiable operations into automatic differentiation frameworks, such as the SoftJAX and SoftTorch libraries ( 18 ) , provide general-purpose continuous relaxations for elemental primitives (e.g., thresholding, fuzzy logic). While these drop-in replacements facilitate the formulation of analytical approximations akin to our continuous baseline, cascading numerous element-wise soft operators through complex topological algorithms frequently exacerbates gradient attenuation and mathematical entanglement. In contrast, our Lipschitz-regularized neural emulation circumvents the need for explicit operator relaxation by learning a holistic, topologically informed mapping that ensures robust structural fidelity and stable gradient flow. Our primary contributions are: (1) proposing a general framework for injecting non-differentiable domain knowledge into training via learned surrogates, compared against standard analytical approximations; (2) introducing the Minkowski image loss, a differentiable objective that penalizes errors in rigid-motion invariants (area, perimeter, connected components); and (3) demonstrating that hard geometric constraints paired with Lipschitz regularization prevents violating geometric principles and overfitting observed in unconstrained baselines. Figure 1: Conceptual overview of the differentiable surrogate training framework. The lack of gradient flow (red connection) prevents the direct optimization of SciML models ( M_{\theta} ) using non-differentiable scientific metrics ( \mathcal{G} ), which typically involve discrete operations like thresholding (step function plot). We overcome this limitation by using knowledge and machine learning techniques to create a differentiable surrogate loss ( \mathcal{L} ) that approximates the target metric. This differentiable loss, with a well-behaved optimization landscape, can then be successfully integrated into the model’s training pipeline (green connection), enabling the direct backpropagation of structurally and physically relevant error signals 2 Methodology 2.1 Approaches to Non-Differentiable Metrics Scientific evaluation often relies on functionals f:\mathcal{X}\rightarrow\mathcal{Y} involving discrete operations such as counting, thresholding, or binning. These operations are inherently non-differentiable; they manifest as piecewise constant functions whose gradient \nabla_{\mathbf{x}}f is zero almost everywhere or undefined at jump discontinuities (Dirac deltas), producing a loss surface of flat plateaus that prevents the backpropagation of error signals required to update an upstream model. Two primary methodologies exist to circumvent this limitation. The first relies on analytical approximation , wherein discrete operations are substituted with continuous, smooth relaxations yielding a closed-form differentiable surrogate. The second leverages neural emulation , employing an artificial neural network to map the input space directly to the target metric; because neural networks are differentiable by construction, this learned surrogate integrates seamlessly into the computational graph. 2.1.1 Analytical Approximation of Non-differentiable Metrics The analytical approximation strategy aims to replace non-differentiable operations with smooth variants that permit stable gradient flow. For example, binary thresholding—mathematically represented by the Heaviside step function H(x) —is commonly relaxed using a logistic sigmoid function \sigma(x/\tau) . Here, \tau>0 acts as a temperature parameter controlling the steepness of the approximation. As \tau\to 0 , the continuous function converges to the exact discrete behavior, but its gradients vanish almost everywhere; conversely, larger values of \tau provide stable gradients at the cost of approximation fidelity. Consequently, \tau is often annealed dynamically during optimization. Similar continuous relaxations are employed for discrete selection operations, such as replacing \arg\max with a differentiable softmax approximation ( 12 ) . While these analytical substitutions preserve the mechanistic interpretability of the original metric, deriving stable, closed-form relaxations for complex, multi-step spatial metrics is not always mathematically possible. 2.1.2 Emulation of Non-Differentiable Metrics: Surrogate Modeling and Lipschitz Regularization To restore differentiability, we approximate the discrete map f with a parameterized neural surrogate f_{\theta}:\mathcal{X}\rightarrow\mathcal{Y} . Minimizing the regression error on static data is insufficient: For f_{\theta} to serve as a stable loss function, it must provide “well-behaved” gradients that are informative across the input domain \mathcal{D} . Gradient Propagation : A critical requirement is non-vanishing input sensitivity. To avoid the “dead zones” associated with rectifying activations (e.g., ReLU where \nabla=0 for x<0 ), we employ smooth, C^{\infty} activation functions—specifically the Gaussian error linear unit ( 10 , GELU;) . Furthermore, to facilitate gradient flow through deep architectures, we utilize residual connections \mathbf{y}=\mathbf{x}+F(\mathbf{x}) , allowing error signals to propagate unimpeded through the identity path. Manifold Smoothing : Early in training, the upstream model may yield a distribution P_{\mathrm{g}} far from the data distribution P_{\text{data}} ; if trained only on P_{\text{data}} ’s support, the emulator can yield arbitrary gradients for \mathbf{x}\sim P_{\mathrm{g}} . We hence inject Gaussian noise \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},\sigma^{2}\mathbf{I}) during emulator training, effectively convolving the target functional with a smoothing kernel and improving robustness to off-manifold queries. (In practice, we found this step unnecessary for the OPERA dataset given the mix-up augmentation described in Section 3.1, and set \sigma^{2}=0 in all

Read the complete text