Systems of ODEs Parameters Estimation by Using Stochastic Newton-Raphson and Gradient Descent Methods

S. Syafiie, Aries Subiantoro, Vivi Andasari, Fernando Tadeo · 2025

Read Systems of ODEs Parameters Estimation by Using Stochastic Newton-Raphson and Gradient Descent Methods by S. Syafiie, Aries Subiantoro, Vivi Andasari, Fernando Tadeo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12856.txt.

Introduction

Syafiie and co-authors fit the hidden parameters of differential equation systems with two classic optimizers, then make both cheaper by adding randomness. Their 2025 paper addresses ODE models of natural processes, where estimating parameters from data is hard when models are nonlinear and high-dimensional. Reformulating the problem through discrete derivatives and Taylor expansion lets Newton-Raphson and gradient descent both apply.

Stochastic versions of each method then handle large systems at reduced cost. Across linear and nonlinear test problems, the new methods beat the standard nonlinear least squares on bias and several error measures. Newton-Raphson converged fastest; gradient descent proved tougher on chaotic systems though sometimes settling short of the best answer. The authors offer the pair as a practical fitting toolkit for the era of big dynamic datasets. Choosing between them becomes a question about the system's temperament: smooth and small, or wild and large.

Catalog record

CREATORS. Syafiie, Aries Subiantoro, Vivi Andasari, Fernando Tadeo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-12856

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Systems of ODEs Parameters Estimation by Using Stochastic Newton-Raphson and Gradient Descent Methods S. Syafiie, Aries Subiantoro, Vivi Andasari, Fernando Tadeo Abstract Ordinary differential equations (ODEs) are widely used to describe the time evolution of natural phenomena across various scientific fields. Estimating the parameters of these systems from data is a challenging task, particularly when dealing with nonlinear and high-dimensional models. In this paper, we propose novel methodologies for parameter estimation in systems of ODEs by using the Newton-Raphson (NR) method and Gradient Descent (GD) method. By leveraging the discrete derivative and Taylor expansion, the problem is formulated in a way that enables the application of both methods, allowing for flexible, efficient solutions. Additionally, we extend these approaches to stochastic versions — Stochastic Newton-Raphson (SNR) and Stochastic Gradient Descent (SGD) — to handle large-scale systems with reduced computational cost. The proposed methods are evaluated by using numerical examples, including both linear and nonlinear parameter models, and compare the results to the well-known Nonlinear Least Squares (NLS) method. While NR converges rapidly to the optimal solution, GD demonstrates robustness in handling chaotic systems, though it may occasionally lead to suboptimal results. Overall, the proposed methods provide improved accuracy in parameter estimation for ODE systems, outperforming NLS in terms of error metrics such as bias, mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination R². These methods offer a valuable tool for fitting ODE models, particularly in scenarios involving big data and complex dynamics. A Preprint Keywords System of ODEs \cdot Data-fitting \cdot S tochastic Newton-Raphson \cdot Stochastic Gradient descent \cdot Nonlinear systems \cdot Parameters estimation 1 Introduction A well presented physical phenomenon mathematically is described by an ODE or a systems of ODEs. Thus, ODEs are a very important powerful tool to formulate, study and analyze physical behavior of a system. The ODEs are extensively employed in science and technology for example in physics, chemistry, biology, mechanics, astronomy, economics, etc. The famous ODE is due the objective laws which governs certain phenomena can be described as an ODE or a system of ODEs. Furthermore, the ODEs solution is given a better future prediction. ODE is an equation having only one instantaneous derivative variable which respects to a single independent variable. Whereas partial differential equation (PDE) is an equation having a single dependent variable which respect to several independent variables. Most physical systems may experience that the previous dependent variable affects the current measurement of dependent variables. This experiencing phenomenon is well described in a delay differential equation (DDE). Also, ODE provides a direct representation of a physical system evolution while allowing the size sample step in irregular space. In an interest for experimental data collection, the sampling interval may vary for each collecting time. Therefore, this article aims to fit a given data set to a system of ODEs model. A curve fitting is a process of identifying parameters of the mathematical models that gives fittest curve to the given data sets. Fitting to a dynamical system is a challenging task. Fitted curves can be implemented as a tool for data visualization to infer the query where no data are available, with possibility to seek value beyond the range of the observed data. An ODE represents the physical system behavior. It is usually described mathematically as \frac{dx}{dt}=f(t,x,a),\quad x(0)=x_{0}, (1) where x is a physical system variable measured at time t , f(\cdot) is the function of the system evolution, a is time-independent parameter and x_{0} is an initial condition. Usually the parameter is estimated based on different of derivative \frac{dx(t)}{dt} to function f(t,x,a) in sum squared error (SSE) of \|\frac{d\hat{x}(t)}{dt}-f(t,\hat{x},\bar{a})\| , where \hat{x} is approximated variable x(t) and \bar{a} is estimated time-independent parameter a . In order to search for an unknown parameter, the ODE in equation ( 1 ) has to be solved or approximated. Most suggested approaches are of proposed explicit solution to the ODE. Based on linearized least square minimization problem of the defined state and measurement vector ( Hwang and Seinfeld, 1972 ) and expanded measurement vector by span state-space to have eigenvectors of weighted matrix ( Gavalas, 1973 ) have been suggested for linear ODEs. Having explicit solutions in term of known elementary or transcendental functions has been suggested by Howland and Vaillancourt (1961) . Also, it has been suggested by Strebel (2023) to determine tangent slope and coordinates for solution of ODE. Adjoint sensitivity analysis is used to estimate ODE solution by Aydogmus and Tor (2021) . As well, it is approximated by introducing integral estimation based on Bayesian collocation approach to avoid expensive numerical solution of the ODE ( Xu et al., 2023 ) . Having ODE as an equality constraint for quadratic programming with selecting minimization problem of measurement vector as a cost function has further suggested by Li et al. (2005) . Using a spline straightforward least squares approach proposed by Varah (1982) , introduced finite set of orthogonal constraints to variational characterization of the solution ( Brunel et al., 2014 ) , penalized splines used for estimating the time-varying coefficients in an ODE based on variation of the nonlinear least squares ( Cao et al., 2012 ) have been suggested. A method for parameter estimation using the Kalman’s filter has been putted into the bargain. With appropriate initial conditions, the filter solution can approximate the minimum-norm weighted least-squares solution of the estimation ( Aidala, 1977 ) . Signals described by linear differential equations whose unknown parameters are estimated by using an orthogonal projection ( Mboup, 2008 ) . Also parameter estimation for ODE has been used a multiple shooting method by Peifer and Timmer (2007) and modification of data smoothing method along with a generalization of profiled estimation by using manifold solution typical dimension of differential equation ( Ramsay et al., 2007 ) . By presented ODE (in equation ( 1 )) in multipoint boundary value problem in discretized for nonlinear constraint optimization problem ( Baake et al., 1992 ) , it has been implied. Parameters estimation for stochastic differential equations by using maximum likelihood and quasi maximum likelihood estimation ( Timmer, 2000 ) , Markov chain Monte Carlo methods ( Mbalawata et al., 2013 ; Chen et al., 2021 ) for linear and non-linear Itô type stochastic differential equations have been hinted. Partially matching empirical expected signature (using Monte Carlo approximation) of the observed path has been suggested to estimate unknown parameters ( Papavasiliou and Ladroue, 2011 ) . Furthermore, comparison study of several estimated method for nonlinear stochastic differential equations with discrete time measurement has been conducted by Singer (2002) . Surveyed and discussed of parameters estimation for stochastic differential equation have been proposed by Nielsen et al. (2000) . Also, interesting in estimating unknown parameters for partial differential equation has been suggested by a parameter cascading method and a Bayesian approach ( Xun et al., 2013 ) . Similar interest is also for estimating unknown parameters for time varying delay differential equation such as using least squares support vector machines ( Mehrkanoon et al., 2014 ) . Parameterisation obtained from the power balance equation of Euler-Lagrange systems can be used for their identification and adaptive control even in a

Read the complete text