Derivative-free discrete gradient methods

Håkon Noren Myhr, Sølve Eidnes · 2026

Read Derivative-free discrete gradient methods by Håkon Noren Myhr, Sølve Eidnes (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07479.txt.

Introduction

Håkon Noren Myhr and Sølve Eidnes work on numerical methods that respect the physics they simulate. Their 2026 preprint concerns discrete gradient methods, integrators that preserve first integrals of ordinary differential equations exactly. Energy-like quantities then stay right even over long runs. The authors aim to build such a method without needing derivatives.

Their construction mixes order theory with the symmetrized Itoh-Abe discrete gradient and finite differences. The result is an integral-preserving method of fourth order that is derivative-free. Avoiding derivatives matters when derivatives are costly or unavailable in closed form. Readers in scientific computing get a careful example of structure-preserving design, where the method honors the equation's own conservation laws.

Catalog record

CREATORHåkon Noren Myhr, Sølve Eidnes
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-07479

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Derivative-free discrete gradient methods Thanks: The research was supported by the Research Council of Norway, through the projects DynNoise (No. 339389) and PhysML (No. 338779). Håkon Noren Myhr, Sølve Eidnes Abstract . Discrete gradient methods are a class of numerical integrators producing solutions with exact preservation of first integrals of ordinary differential equations. In this paper, we apply order theory combined with the symmetrized Itoh–Abe discrete gradient and finite differences to construct an integral-preserving fourth-order method that is derivative-free. The numerical scheme is implicit and a convergence result for Newton’s iterations is provided, taking into account how the error due to the finite difference approximations affects the convergence rate. Numerical experiments verify the order and show that the derivative-free method is significantly faster than obtaining derivatives by automatic differentiation. Finally, an experiment using topographic data as the potential function of a Hamiltonian oscillator demonstrates how this method allows the simulation of discrete-time dynamics from a Hamiltonian that is a combination of data and analytical expressions. 2020 Mathematics Subject Classification Primary 65L99, Secondary 65L12 1. Introduction For an ODE \frac{d}{dt}x=f(x),\quad f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}, (1.1) a first integral is a function G:\mathbb{R}^{n}\rightarrow\mathbb{R} that remains constant along a solution trajectory x(t) of ( 1.1 ), i.e. G(x(t))=G(x(t_{0})) for all t>t_{0} . A first integral could represent conservation laws for physical quantities, such as the energy and angular momentum in the Kepler problem [ 8 , Ch. I.2] or the total mass in a chemical reaction [ 8 , Ch. IV.1] . A particular example of a first integral is the Hamiltonian H:\mathbb{R}^{2d}\rightarrow\mathbb{R} of a canonical Hamiltonian system, which is a system of the form \dot{x}=S\nabla H(x),\quad\text{where}\;S:=\begin{bmatrix}0&I\\ -I&0\end{bmatrix}\in\mathbb{R}^{2d\times 2d}, (1.2) with I being the d\times d identity matrix. That the Hamiltonian is a first integral can be seen by calculating \frac{d}{dt}H(x)=\nabla H(x)^{T}S\nabla H(x)=0. (1.3) This is true for any system \dot{x}=S\nabla H(x) where S is a skew-symmetric matrix, and not only for canonical Hamiltonian systems. In physics, one could think of H(x) as representing the total energy of the system and thus canonical Hamiltonian systems could be used to represent systems with energy conservation. Hamiltonian systems are also used in the setting of computational statistics when sampling from a probability distribution [ 19 ] . Here, the volume preservation (symplecticity [ 8 , Ch. VI] ) of the Hamiltonian dynamics simplifies the sampling procedure and allows for more efficient sampling algorithms called Hamiltonian Monte Carlo [ 19 ] . In this paper, we will study and construct methods that preserve first integrals such as H(x) under discretization in time, while not relying on derivatives of H . 1.1. Discrete gradient methods In some cases, first integrals are preserved by Runge–Kutta methods. For instance, any Runge–Kutta method preserves linear first integrals [ 21 ] and symplectic Runge–Kutta methods preserve quadratic invariants: H(x)=x^{T}Ax when A is square and symmetric matrix [ 2 ] . However, for n>2 , no Runge–Kutta method can preserve all polynomial invariants of degree n [ 1 ] . Discrete gradient methods, on the other hand, are numerical integrators specifically constructed to preserve first integrals of a general form. They were defined formally for the first time in [ 5 ] . A discrete gradient (DG) of a first integral H is a function \overline{\nabla}H:\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n} that satisfies the properties \displaystyle\overline{\nabla}H(x,\hat{x})^{T}(\hat{x}-x) \displaystyle=H(\hat{x})-H(x), \displaystyle\overline{\nabla}H(x,x) \displaystyle=\nabla H(x). Note that, as remarked in [ 4 ] , the second property follows from the first if we require the discrete gradient to be differentiable in the second argument. A discrete gradient method (DGM) is a time-stepping scheme for a skew-gradient system, providing approximations \hat{x}\approx x(t+h) given x\approx x(t) by \hat{x}=x+hS\overline{\nabla}H(x,\hat{x}). (1.4) DGMs preserve invariants since, by the skew-symmetry of S , we have that \displaystyle H(\hat{x})-H(x) \displaystyle=\overline{\nabla}H(x,\hat{x})^{T}(\hat{x}-x) \displaystyle=h\overline{\nabla}H(x,\hat{x})^{T}S\overline{\nabla}H(x,\hat{x})=0, which could be seen as a discrete version of ( 1.3 ). These methods rely on approximations of the gradient \overline{\nabla}H\approx\nabla H , for which there exist several explicitly defined examples. The most common are the midpoint [ 5 ] and average vector field (AVF) [ 9 ] discrete gradients, which are second-order approximations of the gradient, and the Itoh–Abe discrete gradient [ 10 ] , which is of first order. 1.2. Derivative-free numerical algorithms Derivative-free numerical algorithms are useful in the context of optimization when seeking to minimize a function even if the gradient is not available. In optimization research, multiple authors have argued that using finite-difference approximations to replace exact derivatives is prohibitively expensive and sensitive to noise. However, the authors of [ 22 ] demonstrate that finite-difference approximations replacing derivatives within for instance the L-BFGS algorithm are competitive to well-known derivative-free optimization algorithms. Furthermore, optimization libraries such as the Python-based SciPy library [ 24 ] apply finite-difference approximations to approximate Jacobian and Hessian matrices when these are not supplied by the user and needed in specific algorithms. Specifically, finite-difference approximations used to approximate the Jacobian in Newton’s method are discussed in [ 12 , 3 ] . In the setting of numerical integration and preservation of first integrals, the Itoh–Abe discrete gradient represents a derivative-free approximation to the gradient of H(x) . Indeed, it is observed by [ 17 ] that the Itoh–Abe discrete gradient method for gradient systems is equivalent to the derivative-free optimization method, coordinate descent, when applied to solve linear systems. Considering the issue of obtaining samples from probability distributions where direct sampling is difficult, Itoh–Abe discrete gradients have been leveraged within the Hamiltonian Monte Carlo scheme to enable a derivative-free approach [ 14 ] . 2. Derivative-free discrete gradient methods The original discrete gradient method ( 1.4 ) is restricted to have the same order as the discrete gradient \overline{\nabla}H , but higher-order extensions have also been developed [ 16 , 20 ] . A general order theory to construct discrete gradient methods of arbitrary order was presented in [ 4 ] . Replacing S in ( 1.4 ) with an approximation \overline{S}(x,\hat{x},h) allows for the construction of higher-order schemes, even though discrete gradients can at most be second-order approximations of the gradient [ 15 ] . The approximation \overline{S}(x,\hat{x},h) is required to be skew-symmetric and to satisfy \overline{S}(x,x,0)=S(x) . The construction of \overline{S}(x,\hat{x},h) requires potentially higher-order derivatives of the first integral H(x) and of the discrete gradient \overline{\nabla}H(x,\hat{x}) , which might be expensive or infeasible to compute. Automatic differentiation is an increasingly common approach to obtain derivatives of functions, especially in machine learning settings where functions may be outputs of neural networks, but this requires the function to be implemented in a programming language that supports automatic differentiation and could be computationally expensive. In this paper, we propose a fourth-order derivative-free integral-preserving method. This uses the symmetrized Itoh–Abe discrete gradient and an \overline{S}(x,\hat{x},h) that relies on finite difference approximations of derivatives of the discrete gradient \overline{\nabla}H(x,\hat{x}) and the Hessian of H(x) . We will first present a general fourth-order discrete gradient method, derived using the order theory of [ 4 ] . Then we specify the scheme by choosing to use the symmetrized Itoh–Abe discrete gradient and present appropriate finite difference approximations that ensure that the order is kept while the scheme becomes derivative-free. Theorem 2.1 (Fourth-order DGM) . Consider the Hamiltonian system ( 1.2 ). Let \overline{\nabla}H be a second-order symmetric discrete

Read the complete text