Differentiating through Stochastic Differential Equations: A Primer
Rishi Leburu, Levon Nurbekyan, Lars Ruthotto · 2026
Read Differentiating through Stochastic Differential Equations: A Primer by Rishi Leburu, Levon Nurbekyan, Lars Ruthotto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08594.txt.
Introduction
Rishi Leburu, Levon Nurbekyan, and Lars Ruthotto wrote a 2026 arXiv primer on how to take rates of change through equations with noise. The paper was sent to the editors on January 13, 2026. Such equations, called stochastic differential equations, model systems in physics, finance, and economics, and now in machine learning too. A core task is to gauge how a result shifts when a value in the system shifts. Cases in view are air-based values, financial Greeks, and the tuning of neural networks.
Catalog record
| CREATOR | Rishi Leburu, Levon Nurbekyan, Lars Ruthotto |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08594
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Differentiating through Stochastic Differential Equations: A Primer Thanks: Submitted to the editors January 13, 2026. Rishi Leburu, Levon Nurbekyan, Lars Ruthotto Abstract Dynamical systems are essential to model various phenomena in physics, finance, economics, and are also of current interest in machine learning. A central modeling task is investigating parameter sensitivity, whether tuning atmospheric coefficients, computing financial Greeks, or optimizing neural networks. These sensitivities are mathematically expressed as derivatives of an objective function with respect to parameters of interest and are rarely available analytically, necessitating numerical methods for approximating them. While the literature for differentiation of deterministic systems is well-covered, the treatment of stochastic systems, such as stochastic differential equations (SDEs), in most curricula is less comprehensive than the subtleties arising from the interplay of noise and discretization require. This paper provides a primer on numerical differentiation of SDEs organized as a two-tale narrative. Tale 1 demonstrates differentiating through discretized SDEs, known the discretize-optimize approach, is reliable for both Itô and Stratonovich calculus. Tale 2 examines the optimize-discretize approach, investigating the continuous limit of backward equations from Tale 1 corresponding to the desired gradients. Our aim is to equip readers with a clear guide on the numerical differentiation of SDEs: computing gradients correctly in both Itô and Stratonovich settings, understanding when discretize-optimize and optimize-discretize agree or diverge, and developing intuition for reasoning about stochastic differentiation beyond the cases explicitly covered. keywords stochastic differential equations, adjoints, Itô calculus, Stratonovich calculus, automatic differentiation, discretization Funding. This work was supported in part by NSF award DMS 2038118. † † runningheads: Differentiating through Stochastic Differential Equations / R. Leburu, L. Nurbekyan, L. Ruthotto AMS 60H35, 60H10, 65C30, 91G60 1 Introduction Dynamical systems are a natural modeling framework for various applications in physics, economics, finance, and have recently also been used in machine learning. Familiar examples are the Lorenz-63 model for atmospheric convection [ 22 ] , the Black-Scholes model [ 2 , 24 ] for the evolution of a stock price in financial markets, and neural ordinary differential equations [ 3 ] (neural ODEs) for modeling complex nonlinear input-output transformations in machine learning. An important part of modeling with dynamical systems is investigating the sensitivity of these systems with respect to parameters that define their dynamics or initial conditions. For Lorenz-63, for example, one is interested in tuning the system’s coefficients to match meteorological data. For Black–Scholes models, one computes sensitivities to market parameters such as volatility, interest rate, and maturity, as well as the sensitivity of the future stock price with respect to the current price; quantities collectively known as Greeks. Finally, for neural ODEs, one seeks the weights of the networks that define the dynamics that yield the best predictive models. Mathematically, these sensitivities can be expressed as derivatives of an objective function with respect to the parameters of interest. These derivatives are rarely available analytically, necessitating the development of numerical methods for computing them. Thus, the main goal of this paper is to introduce the reader to the basic mathematical and numerical principles underpinning the computation of these sensitivities. In this paper, our aim is to cover the differentiation of stochastic differential equations for the following primary reasons. 1. The literature for deterministic systems is very rich, and there are numerous excellent introductory resources, including textbooks [ 19 ] . Indeed, computing sensitivities of deterministic ODEs with respect to input parameters is part of the standard material in optimal control (OC) theory. 2. The literature on stochastic systems is more limited. Often, standard textbooks on stochastic differential equations (SDEs) [ 12 , 20 , 25 ] or stochastic optimal control [ 5 , 26 , 28 ] do not include discussions of sensitivities akin to those in ODEs. 3. Stochastic systems are more delicate to work with, given various modeling and approximation considerations, such as noise (stochasticity) type and numerical discretization. This delicacy leads to interesting mathematical subtleties that are often overlooked and worth discussing explicitly. 4. Optimal control of stochastic models is an emerging topic of interest in machine learning and applications, so applied mathematicians and domain scientists will benefit from exposure to the topics discussed in the paper. This tutorial serves as an accessible introduction to SDE differentiation, geared toward graduate or advanced undergraduates, faculty, and practitioners who are new to it. As a classroom-style tutorial, this is not a survey covering state-of-the-art methods. We assume readers have background in ordinary differential equations, multivariable calculus, and basic stochastic calculus, as well as some exposure to numerical methods for SDEs [ 12 , 20 , 25 ] . Familiarity with differentiation of ODE-based objectives (the discretize-optimize and optimize-discretize approaches) is helpful but not required. Depending on the course and students’ backgrounds, instructors may consider teaching the differentiation through ODEs before presenting the stochastic case; see [ 19 ] for background on these techniques in the deterministic setting. To keep the material accessible, we have opted for a mostly informal presentation style and small-dimensional numerical examples, referring to rigorous results and more complex examples where suitable. Our presentation draws substantially from the comprehensive treatment in Kidger [ 16 ] , which provides an elegant and abstract framework using rough path theory and controlled differential equations to unify the treatment of ODEs and SDEs. We have adapted this material into a more simplified approach that prioritizes accessibility and intuitive understanding. The paper is organized into two progressive tales, corresponding to the two classic differentiation approaches for SDEs. Tale 1 covers the discretize-optimize approach, showing how to differentiate through numerical schemes for both Itô and Stratonovich SDEs — the two standard frameworks for stochastic calculus. Tale 2 examines the optimize-discretize approach, investigating whether the equations from Tale 1 converge to well-defined continuous processes. Throughout the paper, we validate our theoretical results with numerical experiments using the Black–Scholes model. The code for reproducing the figures in this paper and validating the theoretical results is available at https://github.com/EmoryMLIP/2026-DifferentiatingSDEPrimer . The material naturally fits into courses on stochastic processes, quantitative finance, scientific computing, and machine learning. While standard courses teach SDE simulation, they rarely cover gradient computation despite its importance in applications. This tutorial fills that gap by connecting simulation methods students already know to the techniques needed for optimization and sensitivity analysis. Instructors can use the individual tales as standalone modules or present both tales together for comprehensive coverage. By the end of this primer, readers will understand how to differentiate through SDEs correctly in both Itô and Stratonovich settings using either discretize-optimize or optimize-discretize. We seek to teach readers to recognize why endpoint evaluation matters and understand the fundamental differences between differentiating deterministic and stochastic systems. This foundation enables reasoning about stochastic differentiation problems beyond the cases we explicitly cover. 2 Tale 1. Differentiating through SDEs. Discretize-Optimize We demonstrate the reliability of the discretize-optimize approach for computing gradients through SDEs. We show how to discretize Itô and Stratonovich SDEs using corresponding standard numerical schemes, then differentiate through their chosen respective schemes via automatic differentiation to obtain the desired sensitivities. We adopt a notational convention where boldface is used for matrices to distinguish them from random variables, both of which appear as uppercase letters, and vectors are represented by lowercase letters. This convention naturally helps