Worst-case Nonlinear Regression with Error Bounds

Alberto Bemporad · 2026

Read Worst-case Nonlinear Regression with Error Bounds by Alberto Bemporad (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12334.txt.

Introduction

Alberto Bemporad works on fitting a model to a nonlinear function with firm error bounds. The goal is a stand in model, such as a feedforward neural network. The stand in should stay close to the true function across a whole compact set of inputs. Close here means the worst gap, not just the mean gap.

Bemporad uses an active learning approach to build that stand in. The method picks new points to learn from as it goes. It seeks to shrink the largest error in the fit. Since the error bounds are fixed by math, not by chance, the result comes with a firm pledge. Such pledges matter when a model is used in checks where a bad fit has real cost.

Catalog record

CREATORAlberto Bemporad
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12334

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Worst-case Nonlinear Regression with Error Bounds Alberto Bemporad Abstract This paper proposes an active-learning approach to worst-case nonlinear regression with deterministic error guarantees. Given a known nonlinear function defined over a compact set, we compute a surrogate model, such as a feedforward neural network, by minimizing the maximum absolute approximation error. To address the nonsmooth nature of the resulting minimax problem, we introduce a smooth approximation of the L ∞ L_{\infty} -type loss that enables efficient gradient-based training. We iteratively enrich the training set by actively learning points of largest approximation error through global optimization. The resulting models admit certified worst-case error bounds, either constant or input-dependent, over the entire input domain. The approach is demonstrated through approximations of nonlinear functions and nonconvex sets, as well as through the derivation of uncertain models of more complex nonlinear dynamics within a given model class, and the approximation of explicit model predictive control laws. 1 Introduction Function approximation methods are fundamental in control systems engineering, both to obtain control-oriented models of dynamical systems and to simplify complex nonlinear control laws for online implementation. It can be used either to learn models or control laws directly from data, or to simplify a known model or control function. In both cases, the typical approach is to collect a sufficiently large set of samples and solve a nonlinear regression problem, for example by training a feedforward neural network (NN) by minimizing the mean squared error (MSE), possibly under regularization on the model parameters. In control applications, however, it is often crucial to quantify a bound on the approximation error between the true function, when known, and the learned model. Such bounds are needed, for example, in robust control design to characterize modeling errors treated as additive disturbances, and when approximating control laws to guarantee closed-loop stability and constraint satisfaction despite approximation errors. This issue is particularly relevant for approximating model predictive control (MPC) laws to reduce online computational effort, typically using NNs [ 22 , 12 , 16 ] . However, most existing methods do not guarantee bounds on the worst-case approximation error (WCE) between the exact MPC law and its approximation. The recent contribution [ 1 ] addresses this problem by exploiting Lipschitz continuity, assuming the availability of a sufficiently large training dataset. In system identification for robust control, set-membership methods have been used to obtain models with guaranteed bounded error predictions, including nonlinear models [ 21 ] , often under assumptions on the system dynamics. Similar ideas have been applied to approximate nonlinear MPC laws in [ 11 ] under Lipschitz continuity assumptions. Gaussian processes [ 24 ] offer an alternative by providing models with quantified uncertainty, but they rely on restrictive model classes and yield only probabilistic guarantees. Robustness in nonlinear regression has been studied in the statistics literature mainly to mitigate the effect of outliers in training data [ 14 ] . Here, instead, the goal is to minimize and quantify the WCE over all possible inputs in a given set. This problem is related to uncertainty quantification and conformal prediction [ 2 ] , which typically provide probabilistic confidence intervals rather than worst-case guarantees [ 5 ] . Achieving the latter requires different techniques; for example, in [ 13 ] semidefinite programming is used to certify the robustness of NN predictions against input uncertainties. The standard workflow to obtain nonlinear models with bounded prediction errors is to first identify a model and then estimate its approximation error. However, a small average error, such as a low MSE, does not necessarily imply a small worst-case error. This motivates methods that directly minimize the WCE during training and reliably quantify it afterward. Minimax regression , also known as Chebyshev regression or L_{\infty} estimation [ 23 , Ch. 7] , addresses this need by minimizing the maximum deviation between the true function and its approximation over a dataset or a domain. Despite its potential for robust system identification and control, it has received limited attention in the machine learning and control communities, with most existing results restricted to linear models. This is partly due to the nonsmooth optimization problems it entails and to the strong dependence of the solution quality on the choice of the training samples. 1.1 Contribution In this paper, we consider fitting a parametric model, such as a NN, to a known nonlinear function using an \ell_{\infty} loss over a finite set of samples, with the goal of minimizing the WCE over a compact input set. To enable the use of efficient quasi-Newton methods such as L-BFGS [ 18 ] together with automatic differentiation, we replace the nonsmooth maximum operator with a smooth (but arbitrarily close) approximation and solve the resulting problem. We reduce the dependence of the WCE on training samples by proposing an active-learning strategy that iteratively adds points of maximum approximation error, identified via global optimization. A key difference with respect to active-learning methods for nonlinear regression and classification, which rely on empirical acquisition functions to select potentially informative data [ 27 , 7 ] , is that here the true function can be evaluated at arbitrary inputs during the acquisition process to explicitly identify a point of maximum error. The approach is therefore closely related to corner-case detection via numerical optimization [ 19 ] . In addition, we propose techniques to quantify the WCE over the entire input set after training, in the form of either constant or input-dependent upper and lower bounds on the approximation error. Finally, we show how the proposed approach can be used to learn sets (e.g., minimally conservative convex inner approximations of given nonconvex sets), uncertain (generally nonlinear) models of nonlinear dynamical systems (e.g., derived from physical principles), and approximate explicit MPC laws with guaranteed bounds on the control error introduced by the approximation. 1.2 Notation We denote by [x]_{+}=\max(x,0) the positive part of x\in\mathbb{R} and by \max(v)=\max_{i=1,\ldots,n}(v_{i}) the maximum component of a vector v\in\mathbb{R}^{n} . Given vectors v_{i}\in\mathbb{R}^{n_{i}} , i=1,\ldots,N , n_{i}\geq 1 , we denote by v=\mathop{\rm col}\nolimits(v_{1},\ldots,v_{N})\in\mathbb{R}^{n} their vertical concatenation. Throughout the paper, with a slight abuse of notation, we define the sign function as \mathop{\rm sign}\nolimits(x)=-1 if x\leq 0 and \mathop{\rm sign}\nolimits(x)=1 if x>0 . 2 Minimax Nonlinear Regression Given a nonlinear function f:{\mathcal{X}}\to\mathbb{R} over a compact set {\mathcal{X}}\subset\mathbb{R}^{n} and a class of functions {\mathcal{F}} , our goal is to find a simpler function \hat{f}:{\mathcal{X}}\to\mathbb{R} , \hat{f}\in{\mathcal{F}} , that minimizes the WCE between f(x) and \hat{f}(x) over {\mathcal{X}} , and to quantify such an error. Let \theta be the vector of parameters identifying the function \hat{f} within {\mathcal{F}} , \theta\in\mathbb{R}^{n_{\theta}} . For example, \theta can collect the weights and bias terms of a NN, the coefficients of a polynomial or other basis functions, etc. We want to solve the following problem: \theta^{\star}\in\arg\min_{\theta}\left(r(\theta)+\max_{x\in{\mathcal{X}}}|f(x)-\hat{f}(x;\theta)|\right) (1) where r(\theta) is a regularization term, such as an \ell_{2} or \ell_{1} regularizer, or a penalty introduced to promote certain properties that \hat{f} should have. Solving Problem ( 1 ) poses two difficulties: ( i ) it is a bilevel optimization problem, due to the inner maximization problem over {\mathcal{X}} ; and ( ii ) the inner objective function is non-smooth due to the maximum operator, which makes applying very efficient quasi-Newton methods, such

Read the complete text