SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradients

Anselm Paulus, A. René Geist, Vít Musil, Sebastian Hoffmann, Onur Beker, Georg Martius · 2026

Read SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradients by Anselm Paulus, A. René Geist, Vít Musil, Sebastian Hoffmann, Onur Beker, Georg Martius (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08824.txt.

Introduction

Anselm Paulus and colleagues give gradient-based tuning usable gradients for hard operations. Thresholds, Boolean logic, discrete indexing, and sorting yield zero or undefined gradients in JAX and PyTorch. Their SoftJAX and SoftTorch libraries swap in soft drop-in replacements, from elementwise ops to fuzzy Boolean handling and transport-based sorting, with straight-through estimation throughout.

Catalog record

CREATORAnselm Paulus, A. René Geist, Vít Musil, Sebastian Hoffmann, Onur Beker, Georg Martius
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88221
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08824

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradients Anselm Paulus, A. René Geist, Vít Musil, Sebastian Hoffmann, Onur Beker, Georg Martius Abstract Automatic differentiation (AD) frameworks such as JAX and PyTorch have enabled gradient-based optimization for a wide range of scientific fields. Yet, many “hard” primitives in these libraries such as thresholding, Boolean logic, discrete indexing, and sorting operations yield zero or undefined gradients that are not useful for optimization. While numerous “soft” relaxations have been proposed that provide informative gradients, the respective implementations are fragmented across projects, making them difficult to combine and compare. This work introduces SoftJAX and SoftTorch, open-source, feature-complete libraries for soft differentiable programming . These libraries provide a variety of soft functions as drop-in replacements for their hard JAX and PyTorch counterparts. This includes (i) elementwise operators such as clip \operatorname{clip} or abs \operatorname{abs} , (ii) utility methods for manipulating Booleans and indices via fuzzy logic, (iii) axiswise operators such as sort \operatorname{sort} or rank \operatorname{rank} – based on optimal transport or permutahedron projections, and (iv) offer full support for straight-through gradient estimation. Overall, SoftJAX and SoftTorch make the toolbox of soft relaxations easily accessible to differentiable programming, as demonstrated through benchmarking and a practical case study. Code is available at github.com/a-paulus/softjax and github.com/a-paulus/softtorch . † † affiliationnotice: * Equal contribution. Libraries implemented by Anselm Paulus. ⬇ import softjax as sj arr = jnp . array ([ x , -1.0, 0.5, 0.8, 2.0]) ranks = sj . rank ( arr , method = "neuralsort" , mode = "smooth" )[0] Figure 1 : Top: SoftJAX and SoftTorch provide differentiable surrogates for discrete operations i. e., \operatorname{rank}_{\tau} instead of \operatorname{rank} . Numerous soft approximations can be obtained by tuning the softness parameter \tau , selecting the softening method (e. g., “neuralsort” or “softsort”), and choosing a smoothness mode (e. g., smooth for \mathcal{C}^{\infty} or c1 for \mathcal{C}^{1} ). Bottom: The soft surrogates shown above are instantiated using just three lines of code. 1 Introduction Automatic differentiation (AD) frameworks have enabled rapid progress in machine learning. They make gradient computation efficient, user-friendly, and composable. Thereby, they become a ubiquitous tool widely adopted beyond machine learning. This includes (i) differentiable rendering ( Wang et al. 2019 ) using Nerfs ( Mildenhall et al. 2020 ) and Gaussian splatting ( Kerbl et al. 2023 ) ; (ii) differentiable simulation , such as MuJoCo XLA (MJX) ( Todorov et al. 2012 ; MuJoCo 2024 ; Paulus et al. 2025 ) and WARP ( Howell 2025 ) ; (iii) structured prediction like ranking ( Rolínek et al. 2020a ) and matching ( Rolínek et al. 2020b ) ; (iv) combinatorial layers for CEM ( Amos & Yarats 2020 ) , MPC ( Amos et al. 2018 ) , and discrete decisions ( Pogancic et al. 2020 ; Berthet et al. 2020 ) ; (v) differentiable optimization ( Amos & Kolter 2017 ; Agrawal et al. 2019 ; Blondel et al. 2022 ) ; and (vi) physical simulations , e. g., for a gravitational wave detector ( Ruiz-Gonzalez et al. 2025 ) . However, the programs for these and many more applications contain classical operations using comparisons for branching code, ranking elements, and so forth. Unfortunately, AD does not necessarily result in informative gradients. Here, informative means providing a stable local direction leading to improvements in the objective. Uninformative are cases with zero gradients (e. g., rounding, comparisons, indexing, sort/max/top-k/median) and arbitrary subgradients (ReLU at zero, sort with duplicates). Many applications call for replacing such hard operations with soft ones that yield informative gradients. A successful example is addressing the “dying ReLU problem” ( Lu et al. 2020 ) , where ReLU’s zero gradients hinder gradient-based optimization of deep neural networks, by replacing the ReLU function with a smooth relaxation such as SiLU or Softplus. While many relaxation techniques have been proposed, the existing toolbox is fragmented. Proposed techniques include analytic smooth surrogates (e. g., SiLU for ReLU ( Hendrycks & Gimpel 2016 ) , sigmoid for steps ( Petersen et al. 2021 ; Petersen et al. 2022 ) , and softmax for argmax ( Beker et al. 2025 ; Beker et al. 2026 ) ), optimization-based approaches (e. g., sorting via regularized optimal transport ( Cuturi et al. 2019 ; Xie et al. 2020 ; Blondel et al. 2018 ) or projections onto the permutahedron ( Blondel et al. 2020 ; Sander et al. 2023 ) , proximal methods ( Paulus et al. 2024 ) ), stochastic relaxations (e. g., Gumbel-softmax ( Maddison et al. 2017 ; Jang et al. 2017 ; Paulus et al. 2020 ) , perturbed optimizers ( Berthet et al. 2020 ; Niepert et al. 2021 ) ), gradient “hacks” / estimators (straight-through ( Bengio et al. 2013 ; Sahoo et al. 2023 ) , black-box differentiation ( Pogancic et al. 2020 ) ) and more. To unify soft relaxations and ease their use, we present SoftJAX and SoftTorch . Our libraries provide soft drop-in replacements for many commonly used hard operations in JAX ( Bradbury et al. 2018 ) and Pytorch ( Paszke et al. 2019 ) . In turn, SoftJAX and SoftTorch form easy to use abstraction layers between user code and underlying primitive operations. Our treatment also draws unifying connections, generalizes various techniques, and provides comprehensible “softness” knobs and “mode” families across operations. As an example, see Figure 1 where we plot the soft \operatorname{rank} operator for varying softness parameters \tau>0 and softening modes. As for all our relaxations, we recover the original hard operation as \tau\rightarrow 0^{+} . By combining the vast toolbox of soft relaxations, our libraries make soft differentiable programming accessible. We next explain the operators implemented in SoftJAX and SoftTorch and how their hard versions are relaxed to soft counterparts. Section 2 reviews the basics of soft surrogates and straight-through estimation. Section 3 then describes the softening of elementwise operators in SoftJAX and SoftTorch, such as \operatorname{sign} or \operatorname{round} , via relaxations of the Heaviside step function. Section 4 shows how more involved axiswise operators, such as \operatorname{sort} or \operatorname{quantile} , can be softened via optimal transport or projections onto the unit simplex or the permutahedron. An overview of all currently implemented functions in SoftJAX is available in Figure 9 . Finally, Section 5 presents runtime and memory comparisons to help end users select suitable methods that balance the trade-offs required by their applications. 2 Softening and Straight-through estimation Many functions used in modern ML frameworks are poorly suited for automatic differentiation as they are discontinuous or have large regions with zero derivative. To resolve this, our libraries have one main goal: to provide informative gradients for any program written in the supported frameworks. The underlying mechanism rests on two core concepts: soft surrogates and straight-through estimation . Soft surrogate. Soft surrogates replace the original function with another function that has more informative gradients for optimization. Specifically, we call the function f_{\tau} with softening parameter \tau>0 a soft surrogate of f if (i) f_{\tau} is continuous and differentiable almost everywhere, (ii) f_{\tau} yields informative gradients over the relevant domain, i. e., avoids extended regions of zero derivative, and (iii) f_{\tau} recovers f in the limit \tau\to 0^{+} . The

Read the complete text