Derivative Estimation from Coarse, Irregular, Noisy Samples: An MLE–Spline Approach

Konstantin E. Avrachenkov, and, Leonid B. Freidovich · 2025

Read Derivative Estimation from Coarse, Irregular, Noisy Samples: An MLE–Spline Approach by Konstantin E. Avrachenkov, and, Leonid B. Freidovich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-22176.txt.

Introduction

Avrachenkov and Freidovich estimate slopes from coarse, uneven, noisy samples with a likelihood and spline method, in a paper from twenty twenty-five. Many sensors report rarely and at irregular times, with noise on each read. Finding the rate of change from such data is hard. Classic observers struggle when sampling is coarse and noise is high.

Their estimator caps the size of a higher slope to keep fits stable. The solution takes the form of a smooth spline curve. A fresh parameterization yields fast online updates as data arrive. Two variants trade smoothness against computing speed. Simulations beat high-gain observers and twisting methods under coarse, noisy sampling. The gain helps systems that cannot sample faster.

Catalog record

CREATORKonstantin E. Avrachenkov, and, Leonid B. Freidovich
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-22176

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Derivative Estimation from Coarse, Irregular, Noisy Samples: An MLE–Spline Approach Konstantin E. Avrachenkov, and, Leonid B. Freidovich Abstract We address numerical differentiation under coarse, non-uniform sampling and Gaussian noise. A maximum-likelihood estimator with L 2 L_{2} -norm constraint on a higher-order derivative is obtained, yielding spline-based solution. We introduce a non-standard parameterization of quadratic splines and develop recursive online algorithms. Two formulations—quadratic and zero-order—offer tradeoff between smoothness and computational speed. Simulations demonstrate superior performance over high-gain observers and super-twisting differentiators under coarse sampling and high noise, benefiting systems where higher sampling rates are impractical. Keywords: Calculus of variations, Maximum likelihood estimation (MLE), Non-uniform sampling, Numerical differentiation, Optimization methods, Splines 1 Introduction Control systems can benefit from feedback based on estimates of the derivatives of measured signals. Numerical differentiation is a well-established area of research focused on estimating the derivative of a continuous-time signal from imprecisely measured sampled values. In this context, sampling refers to obtaining values of a continuously differentiable function at specific moments, where the measurements are contaminated by noise, as is typical for sensor data. This process inherently introduces both measurement noise and discretization effects. Our objective is to estimate the derivative of the sampled signal by exploiting its inherent properties, such as smoothness and known bounds on its derivatives, while incorporating assumptions about the probability distribution of the additive measurement noise. In comparison to existing methods, we aim to develop and implement a technique capable of addressing coarse, non-uniform sampling and unbounded noise. Our method is based on the maximum likelihood approach directly estimating the derivative under L_{2} -norm constraint on the value of a higher order derivative and the recursive implementation of the spline solution, providing a framework for optimal online estimation of derivatives in non-asymptotic scenarios. This makes it particularly valuable when reducing sampling intervals is impractical due to hardware limitations or other constraints. Given the similarity between the spectra of a signal and its derivative, reconstructing the derivative of a signal becomes fundamentally infeasible if the sampling frequency is too low, as dictated by the classical Nyquist–Shannon sampling theorem. Moreover, there are well-known fundamental limitations on achievable numerical differentiation accuracy, as discussed in [ 1 , 2 ] . The derivative of a signal reveals its trend and helps to predict future values. However, forecasting the output of a dynamical system with an unknown model, subject to uncertainty and disturbances, is inherently challenging without restrictive and often impractical assumptions. This difficulty is further exacerbated because a smooth signal corrupted by white noise is mathematically non-differentiable. A rigorous way to address this issue is to reformulate the differentiation problem—either by introducing notions of robustness and asymptotic exactness, as in [ 3 ] and many subsequent works, or by casting it as a regularized optimization problem. To the best of our knowledge, the widely used approaches to numerical differentiation include the following: • Linear continuous-time filters approximating the differentiation operation and their discrete-time implementations, Euler or higher-order finite differences, Savitzky–Golay filters [ 4 , 5 ] , Kalman filters [ 6 , 7 ] , and linear and nonlinear high-gain observers [ 8 , 9 , 10 , 11 ] ; • Sliding-mode-based differentiators and observers [ 3 , 12 , 13 , 14 ] , based on the idea of achieving insensitivity to perturbations via dynamic collapse, including Levant differentiator, also known as super-twisting differentiator, and their discrete-time implementations; • Algebraic differentiators [ 15 , 16 , 17 ] ; and • Optimization-based algorithms using Tikhonov regularization of ill-posed problems [ 18 , 19 ] . The latter approach is closely related to ours. However, there are important differences. In the vast majority of works using Tikhonov regularization, it is proposed first to estimate the signal and then to differentiate the obtained smooth solution. As will be seen, we obtain the derivative directly, which results in significant savings in terms of the number of parameters. Our main interest lies in targeted applications for control systems, which require online differentiation of imperfectly measured sampled signals using only current or past values. For some applications, such as those under consideration, recursive or online methods are of paramount importance, while the majority of published results on Tikhonov regularization approach (and on many other approaches as well) in the applied mathematics domain focus on non-recursive estimation over a finite fixed time interval. It should be clarified that the Kalman filter is not explicitly designed as a differentiator. Both the high-gain observer (HGO) and the Kalman filter share the structure of a Luenberger observer [ 20 ] : a dynamical system designed to estimate the states of a model using input and output signals. The gains for the HGO are assigned through a specialized pole-placement technique [ 21 , 22 , 23 ] to achieve fast error dynamics, whereas the gains for the Kalman filter are determined using the covariance matrices of (Gaussian) process and measurement noise. Despite this difference, both schemes are frequently used to estimate the state of a simple model structured as a chain of integrators, effectively enabling the estimation of derivatives through state observers with a significantly simplified model of the unknown process dynamics. Levant’s online differentiator [ 3 ] , also known as super-twisting differentiator, has been shown to achieve optimal asymptotic precision with respect to the size of the sampling intervals, under the assumptions and definitions outlined in [ 1 ] . However, in practical applications, improving numerical accuracy by reducing noise magnitude and sampling interval size is often impractical. Such improvements typically require more advanced hardware and larger memory storage, leading to higher computational demands and increased delays due to longer computation times. In many cases, trial-and-error parameter tuning becomes unavoidable instead. Consequently, adopting alternative schemes that are optimal under realistic constraints, with a small number of tunable parameters, can be highly advantageous in practice. Various methods and discretization schemes for continuous-time numerical differentiators have been proposed and studied. However, to the best of our knowledge, most analyses have been carried out under two restrictive assumptions: bounded noise and continuous measurements or uniform sampling with a bounded time-step size, as discussed in [ 13 , 24 , 25 , 26 , 27 ] . The goal of the present work is to address these two shortcomings. We emphasize that our solution is optimal in the non-asymptotic sense, that is, even for coarse sampling. In addition to the contributions already mentioned, we build on several key insights and extensions. First, in many works, cubic smoothing splines have been employed, where one first constructs a cubic smoothing spline for the coarsely measured signal and then differentiates it to obtain an estimate of the signal’s derivative, as e.g. in [ 28 ] . However, this approach requires estimating twice as many parameters compared to using quadratic splines directly to approximate the derivative. Second, we introduce a non-standard parametrization of quadratic splines, which better suits the nature of the problem. Third, we emphasize a truly recursive, online nature of our approach: in our setting, unlike standard recursive least-squares, the number of parameters to be identified increases simultaneously with the acquisition of new observations. Fourth, we demonstrate that even zero-order splines work effectively, and they offer an advantage

Read the complete text