Gradient-based optimization of exact stochastic kinetic models
Francesco Mottes, Qian-Ze Zhu, Michael P. Brenner · 2026
Read Gradient-based optimization of exact stochastic kinetic models by Francesco Mottes, Qian-Ze Zhu, Michael P. Brenner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14183.txt.
Introduction
Francesco Mottes, Qian-Ze Zhu, and Michael Brenner tune exact stochastic kinetic models with gradient methods in 2026. Such models describe systems across biology, chemistry, and physics. In them, events are discrete and counts are small, so smooth approximations fall short. Design and inference both need search over paths drawn by the standard sampler.
The discrete reaction events in those paths block the usual gradient flow. A gradient needs a smooth slope, and a jump has none. Their method finds a way to push gradients through such jumps. That opens exact models to the same fast tuning used in deep learning. The method runs on data drawn from past runs, with no fresh shots on the rig.
Catalog record
| CREATOR | Francesco Mottes, Qian-Ze Zhu, Michael P. Brenner |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-14183
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gradient-based optimization of exact stochastic kinetic models Francesco Mottes, Qian-Ze Zhu, Michael P. Brenner Abstract Stochastic kinetic models describe systems across biology, chemistry, and physics where discrete events and small populations render deterministic approximations inadequate. Parameter inference and inverse design in these systems require optimizing over trajectories generated by the Stochastic Simulation Algorithm, but the discrete reaction events involved are inherently non-differentiable. We present an approach based on straight-through Gumbel-Softmax estimation that maintains exact stochastic simulations in the forward pass while approximating gradients through a continuous relaxation applied only in the backward pass. We demonstrate robust performance on parameter inference in stochastic gene expression, accurately recovering kinetic rates of telegraph promoter models from both moment statistics and full steady-state distributions across diverse and challenging parameter regimes. We further demonstrate the method’s applicability to inverse design problems in stochastic thermodynamics, characterizing Pareto-optimal trade-offs between non-equilibrium currents and entropy production. The ability to efficiently differentiate through exact stochastic simulations provides a foundation for systematic inference and rational design across the many domains governed by continuous-time Markov dynamics. Introduction Stochastic kinetic models, formalized as continuous-time Markov processes describing discrete state transitions, arise throughout the quantitative sciences. In systems biology, they are used to describe transcriptional bursting and regulatory switching underlying cell-to-cell variability in gene expression 8 ; 34 . Classic examples include the bistable lysis-lysogeny decision of lambda phage 3 and stochastic induction of the lac operon 29 . In non-equilibrium statistical mechanics, analogous frameworks describe molecular motors, driven transport processes, and the thermodynamic costs associated with maintaining currents far from equilibrium 37 ; 19 . In chemistry and ecology, stochastic kinetics govern reaction networks and population dynamics in regimes where small molecular counts and discrete events render deterministic approximations inadequate 11 ; 6 . The Chemical Master Equation (CME) provides the mathematical framework for describing the time evolution of probability distributions over discrete system states in stochastic kinetic models 42 . Analytical solutions, however, are available only for restricted classes of reaction networks 36 ; 38 . For general systems, exact sample trajectories can be generated using Gillespie’s Stochastic Simulation Algorithm (SSA) 10 , which produces realizations of the underlying continuous-time Markov process. Parameter inference and inverse design in these systems therefore relies mostly on simulation-based approaches. Solving inverse problems in stochastic kinetic systems is essential for extracting mechanistic insight from noisy experimental data, and substantial effort has been devoted to developing inference methods. For systems with state spaces amenable to controlled truncation, Finite State Projection and related methods enable likelihood-based inference 25 ; 26 ; 27 ; 9 , but require explicit representation of the truncated state space. Moment-based methods offer computational efficiency, but only when low-order statistics are informative 43 . Bayesian frameworks, including Approximate Bayesian Computation 41 ; 22 and particle Markov chain Monte Carlo 13 ; 2 , provide rigorous uncertainty quantification but can become computationally demanding in higher-dimensional parameter spaces. Gradient-based optimization is an attractive alternative, since reverse-mode automatic differentiation provides gradients at cost comparable to a single forward evaluation 4 . However, the discrete reaction selections and state updates in the SSA are intrinsically non-differentiable, precluding direct pathwise differentiation. Likelihood ratio estimators yield unbiased gradients without requiring differentiable dynamics 32 ; 15 , but exhibit variance that grows with trajectory length. Finite difference methods provide an alternative, but their computational cost scales linearly with the number of parameters, even when using variance-reduction schemes that drastically improve convergence 1 . Recent work introduced a continuous SSA relaxation enabling end-to-end differentiation 35 , though at the cost of approximation errors in forward dynamics that accumulate over trajectory length, propagate into gradient estimates, and break permutation symmetry across reaction channels. Here we introduce a reparameterized gradient estimator that enables efficient optimization while maintaining exact stochastic simulations. The approach preserves full fidelity of the Chemical Master Equation, retaining discrete reaction events and correct waiting time statistics in forward trajectories, while providing low-variance gradient estimates compatible with modern adaptive optimizers. We achieve this through straight-through Gumbel-Softmax estimation 18 ; 23 , decoupling forward sampling from backward differentiation. In the forward pass, exact discrete samples are drawn according to the SSA. In the backward pass, gradients are propagated through a continuous relaxation of the categorical sampling operation. The resulting estimator exhibits substantially lower variance than likelihood ratio methods and avoids the approximation errors introduced by continuous relaxations of forward dynamics. We demonstrate the framework on two problem classes spanning biophysics and non-equilibrium statistical mechanics. First, we address parameter inference in stochastic gene expression, recovering kinetic rates of telegraph promoter models from both moment statistics and full steady-state probability distributions. Using differentiable histogram construction, we show that distribution-level optimization accurately recovers ground-truth parameters across kinetic regimes characteristic of prokaryotic transcription. Second, we apply the method to optimization problems in stochastic thermodynamics, maximizing non-equilibrium currents under constraints on entropy production and kinetic resources. We characterize Pareto-optimal trade-offs between current magnitude and dissipation, recovering known theoretical bounds. These applications illustrate the generality of the approach: any scalar objective computable from stochastic trajectories becomes accessible to efficient gradient-based optimization. A Reparameterized Pathwise Gradient Estimator The Stochastic Simulation Algorithm involves two sampling operations at each step, selecting which reaction occurs and when it occurs (Fig. 1 , left). Both can be written as deterministic functions of parameter-independent random draws, enabling gradients to flow through the simulation while preserving exact discrete dynamics (Fig. 1 , right). For waiting times, this reparameterization is standard. If u\sim\mathcal{U}(0,1) , then \Delta t=-\log u/a_{0}(\theta) is exponentially distributed with rate equal to the total propensity a_{0}(\theta)=\sum_{j}a_{j}(\theta) . Since \Delta t is an explicit function of the propensities, gradients flow through the waiting time automatically. The categorical sampling of reaction identity appears more problematic, since selecting reaction index r with probability \pi_{r}=a_{r}/a_{0} requires a discrete choice. However, this too admits reparameterization through the Gumbel-Max trick. If u_{k}\sim\mathcal{U}(0,1) are independent samples then g_{k}=-\log(-\log u_{k}) follow the standard Gumbel distribution, and y=\text{one\_hot}\left(\underset{k}{\arg\max}\left(g_{k}+\log\pi_{k}\right)\right) samples from the correct categorical distribution. Expressing the selected reaction as a one-hot vector allows the state update to be written as a matrix-vector product with the stoichiometry matrix. The discrete outcome is now expressed as a deterministic function of fixed random inputs \{g_{k}\} and the parameter-dependent reaction propensities \{a_{k}(\theta)\} . Figure 1: Modeling and Optimization Framework. SSA trajectories are generated exactly in the forward pass by sampling waiting times and discrete reaction events (Left, blue shaded box). In the backward pass, gradients are computed by directly reparameterizing waiting times and applying a continuous Gumbel-Softmax relaxation to reaction selection (Right, yellow shaded box), enabling efficient gradient-based optimization without approximating the stochastic dynamics. Figure 2: Moment matching inferences from synthetic data. Inference of parameters from mean and variance of the true underlying generative model at steady state, for 25 different parameter sets. Gray markers indicate the random initializations. (a) Two parameters of the telegraph model ( k_{\text{on}} , k_{\text{tx}} ) are simultaneously estimated. (b) Mean and standard deviation of the RNA levels at steady state for the true and optimized models. (c) Comparison between the estimated and true rates. The Gumbel-Max formulation explicitly exposes the source of non-differentiability: the \arg\max operation has zero gradients almost everywhere. The straight-through Gumbel-Softmax (ST-GS) estimator resolves this by employing