Nonparametric local polynomial regression for functional covariates
Moritz Jirak, Alois Kneip, Alexander Meister, Mario Pahl · 2025
Read Nonparametric local polynomial regression for functional covariates by Moritz Jirak, Alois Kneip, Alexander Meister, Mario Pahl (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05819.txt.
Introduction
Moritz Jirak and three co-authors study regression when the input is a whole curve, in a 2025 math paper. Such curve-valued inputs live in an endless-dimensional space, which makes fitting a response far harder than with plain numbers. Their tool fits a local polynomial around each target curve.
The method needs a chosen basis and several tuning knobs. The authors bound its error and show it can reach polynomial rates of accuracy when the true response is smooth enough and the knobs are set well. Such rates were long thought out of reach for this problem without stiff extra assumptions like a straight-line response.
Catalog record
| CREATOR | Moritz Jirak, Alois Kneip, Alexander Meister, Mario Pahl |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05819
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Nonparametric local polynomial regression for functional covariates Moritz Jirak, Alois Kneip, Alexander Meister, Mario Pahl Abstract We consider nonparametric regression with functional covariates, that is, they are elements of an infinite-dimensional Hilbert space. A locally polynomial estimator is constructed, where an orthonormal basis and various tuning parameters remain to be selected. We provide a general asymptotic upper bound on the estimation error and show that this procedure achieves polynomial convergence rates under appropriate tuning and supersmoothness of the regression function. Such polynomial convergence rates have usually been considered to be non-attainable in nonparametric functional regression without any additional strong structural constraints such as linearity of the regression function. Keywords : functional data analysis; kernel estimation; nonparametric regression; polynomial convergence rates. MSC2020 classification : 62R10; 62G08. 1 Introduction We consider the nonparametric regression model where one observes the i.i.d. data (X_{j},Y_{j}) , j=1,\ldots,n , given as Y_{j}\,=\,g(X_{j})+\varepsilon_{j}\,. (1.1) The covariates X_{j} are functional random variables and take on their values in the Hilbert space L_{2}([0,1]) , which is endowed with its Borel \sigma -algebra. The regression errors \varepsilon_{j} are conditionally centered given X_{j} and have a finite conditional variance given X_{j} , which is bounded from above by some deterministic constant \sigma^{2}>0 . The goal is to estimate the smooth regression function g , which maps from L_{2}([0,1]) to \mathbb{R} . A famous simplification of this model is functional linear regression where g is imposed to be linear, i.e. g(x)=\langle\theta,x\rangle with L_{2}([0,1]) -inner product \langle\cdot,\cdot\rangle and target function \theta\in L_{2}([0,1]) . Then the statistical problem reduces to the estimation of \theta . The model has been widely studied, e.g. for nonparametric estimation of \theta with optimal convergence rates, we refer to [ 14 ] , to [ 8 ] for adaptive estimation, to [ 4 ] for optimal prediction, to [ 16 ] for asymptotic equivalence to a white noise inverse problem and to [ 5 ] for central limit theorems. Despite the linear structure, polynomial rates slower than the standard parametric rate occur since the covariates take their values in an infinite-dimensional function space. For approaches beyond linearity, we mention quadratic functional regression, see [ 20 ] and fully nonparametric functional regression models (see e.g. the book of [ 13 ] , the review paper of [ 15 ] and the references therein). While convergence rates are usually not provided explicitly, polynomial rates have not been attained in the literature so far. The most commonly used estimation method is the functional data version of the kernel regression estimator (Nadaraya-Watson estimator). Besides, there are extensions from this locally constant estimator to locally linear procedures, see e.g. [ 1 ] , [ 2 ] and [ 3 ] . In the current work, we introduce a locally polynomial estimator for an arbitrary degree. It generalizes the standard local polynomial regression estimator (see e.g. [ 12 ] ) to treat functional covariates. We mention approaches to nonparametric density estimation for functional data, i.e. the wavelet-based method of [ 7 ] and orthogonal series and kernel estimators by [ 9 ] and [ 10 ] for specific diffusion processes. While upper bounds on the convergence rates are provided under quite abstract conditions in these papers, [ 11 ] study artificially contaminated functional data with a Wiener density and attain polynomial convergence rates by series estimators. Under finite smoothness levels of the target function in functional data analysis, only logarithmic or at least sub-polynomial convergence rates are achieved (for lower bounds see [ 18 ] , [ 6 ] and [ 19 ] ). However, the infinite dimensional Gaussian mixtures considered in [ 11 ] are supersmooth in a sense that the functional densities are infinitely-fold differentiable. In the current paper we show that, for functional regression functions under more general supersmoothness constraints (apart from Gaussian mixtures), polynomial convergence rates can be attained by our locally polynomial estimator under specific selection of an orthonormal basis and two tuning parameters denoted by J and K . In Section 2 , we introduce our estimation procedure, in Section 3 we deduce a general asymptotic upper bound on the pointwise estimation error of our method in Theorem 3.1 . This results is used in Section 4 in order to derive polynomial convergence rates. The proofs are deferred to Section 5 . 2 Methodology Let us assume that g is K -fold Fréchet differentiable in some neighbourhood {\cal N}:={\cal N}(x,\delta) of x\in L_{2}([0,1]) with radius \delta>0 where x is the site at which g should be estimated. The K th order Fréchet derivative is supposed to be continuous on {\cal N} . We define \tilde{g}_{j}(\lambda)\,:=\,g\big(\lambda X_{j}+(1-\lambda)x\big)\,. Under the stipulation that X_{j}\in{\cal N} , we apply Taylor approximation to \tilde{g} around \lambda=0 , which yields that \displaystyle g(X_{j}) \displaystyle\,=\,\tilde{g}_{j}(1)=\sum_{k=0}^{K-1}\frac{1}{k!}\tilde{g}_{j}^{(k)}(0)\,+\,{\cal R}_{S}^{(K)}(x,X_{j}) \displaystyle\,=\,\sum_{k=0}^{K-1}\frac{1}{k!}g^{(k)}(x;X_{j}-x,\ldots,X_{j}-x)\,+\,{\cal R}_{S}^{(K)}(x,X_{j})\,, (2.1) where g^{(k)}(x;\cdots) denotes the k th order Fréchet derivative of g at x , which is viewed as a symmetric multilinear map of degree k ; and the remainder term satisfies \big|{\cal R}_{S}^{(K)}(x,X_{j})\big|\,\leq\,\frac{1}{K!}\cdot\sup_{y\in{\cal N}}\big\|g^{(K)}(y;\cdot)\big\|\cdot\delta^{K}\,, (2.2) with \big\|g^{(K)}(y;\cdot)\big\|\,:=\,\sup\big\{\big|g^{(K)}(y;u_{1},\ldots,u_{K})\big|\,:\,\|u_{1}\|=\cdots=\|u_{K}\|=1\big\}\,. (2.3) Let \{\varphi_{j}\}_{j\in\mathbb{N}} be some orthonormal basis of L_{2}([0,1]) , which is considered as deterministic at this stage. We replace X_{j}-x by its finite-dimensional proxy \sum_{l=1}^{J}\langle X_{j}-x,\varphi_{l}\rangle\varphi_{l} , for some integer J>0 , where \langle\cdot,\cdot\rangle stands for the L_{2}([0,1]) -inner product, so that \displaystyle g(X_{j}) \displaystyle={\cal P}_{J,K}\big(\langle X_{j}-x,\varphi_{1}\rangle,\ldots,\langle X_{j}-x,\varphi_{J}\rangle\big)+{\cal R}_{D}^{(K,J)}(x,X_{j})+{\cal R}_{S}^{(K)}(x,X_{j})\,, (2.4) where \displaystyle{\cal P}_{J,K}\big(\langle X_{j}-x,\varphi_{1}\rangle,\ldots,\langle X_{j}-x, \displaystyle\varphi_{J}\rangle\big) \displaystyle=\sum_{k=0}^{K-1}\frac{1}{k!}\sum_{l_{1}=1}^{J}\cdots\sum_{l_{k}=1}^{J}g^{(k)}(x;\varphi_{l_{1}},\ldots,\varphi_{l_{k}})\cdot\prod_{q=1}^{k}\langle X_{j}-x,\varphi_{l_{q}}\rangle \displaystyle=\sum_{k\in{\cal K}}\xi_{j;{\bf k}}\cdot G_{\bf k}(x)\,, (2.5) with the notation \displaystyle{\cal K} \displaystyle\,:=\,\big\{{\bf k}=(k_{1},\ldots,k_{J})\in\mathbb{N}_{0}^{J}\,:\,k_{1}+\cdots+k_{J}\leq K-1\big\}\,, \displaystyle\xi_{j;{\bf k}} \displaystyle\,:=\,\prod_{l=1}^{J}\langle X_{j}-x,\varphi_{l}\rangle^{k_{l}}\,, \displaystyle G_{\bf k}(x) \displaystyle\,:=\,g^{(|{\bf k}|)}(x;\underbrace{\varphi_{1},\ldots,\varphi_{1}}_{k_{1}-\mbox{fold}},\ldots,\underbrace{\varphi_{J},\ldots,\varphi_{J}}_{k_{J}-\mbox{fold}})\,\big/\,\prod_{j=1}^{J}k_{j}!\,, (2.6) where |{\bf k}|=\sum_{j=1}^{J}k_{j} . Note that {\cal P}_{J,K} forms a J -variate polynomial of degree \leq K-1 and that {\cal R}_{D}^{(K,J)}(x,X_{j}) represents the error coming from the finite-dimensional approximation. It can be bounded from above as follows Lemma 2.1 . On the event X_{j}\in{\cal N} it holds that \big|{\cal R}_{D}^{(K,J)}(x,X_{j})\big|\,\leq\,\sum_{k=1}^{K-1}\frac{\delta^{k-1}}{(k-1)!}\cdot\big\|g^{(k)}(x;\cdot)\big\|\cdot\Big\{\sum_{l>J}\langle X_{j}-x,\varphi_{l}\rangle^{2}\Big\}^{1/2}\,. As the estimation procedure we suggest to consider that polynomial \hat{{\cal P}}_{J,K} which minimizes the contrast functional \tilde{{\cal P}}_{J,K}\,\mapsto\,\sum_{j=1}^{n}\big|Y_{j}-\tilde{{\cal P}}_{J,K}\big(\langle X_{j}-x,\varphi_{1}\rangle,\ldots,\langle X_{j}-x,\varphi_{J}\rangle\big)\big|^{2}\cdot 1_{{\cal N}}(X_{j})\,, among all J -variate polynomials \tilde{{\cal P}}_{J,K} of the degree \leq K-1 . Following the standard strategy in local polynomial regression, the constant coefficient of \hat{{\cal P}}_{J,K} can be employed to estimate g(x) . The scheme is formalized as follows: Write \|z\|_{\cal N}^{2}:=\sum_{j=1}^{n}z_{j}^{2}\cdot 1_{{\cal N}}(X_{j}) , Y:=(Y_{1},\ldots,Y_{n})^{\dagger} and \tilde{{\cal P}}_{J,K}\big(\langle X_{j}-x,\varphi_{1}\rangle,\ldots,\langle X_{j}-x,\varphi_{J}\rangle\big)=\sum_{{\bf k}\in{\cal K}}\,\alpha_{{\bf k}}\cdot\xi_{j;{\bf k}}\,. Then the goal is to select \alpha=(\alpha_{{\bf k}})_{{\bf k}} such that \|Y-\xi\alpha\|_{{\cal N}}^{2} with \xi:=\{\xi_{j,{\bf k}}\}_{j=1,\ldots,n;{\bf k}\in{\cal K}} is minimized. By usual least-square arguments, we deduce that \hat{\alpha} is a minimizing vector if it satisfies the following system of linear equations {\cal M}\hat{\alpha}\,=\,{\cal Y}\,, where \displaystyle{\cal Y} \displaystyle\,=\,\Big(\sum_{j=1}^{n}1_{\cal N}(X_{j})\xi_{j;{\bf k}}\cdot Y_{j}\Big)_{{\bf k}\in{\cal K}}\,, \displaystyle{\cal M} \displaystyle\,=\,\Big(\sum_{j=1}^{n}1_{\cal N}(X_{j})\cdot\xi_{j;{\bf k}}\cdot\xi_{j;{\bf k}^{\prime}}\Big)_{{\bf k},{\bf k}^{\prime}\in{\cal K}}\,. This motivates the definition of the functional local polynomial estimator \hat{g}(x)\,=\,{\bf e}_{0}^{\dagger}({\cal M}+{\cal S})^{-1}{\cal Y}\,, (2.7) for j th unit vector {\bf e}_{j} and the diagonal {\cal K}\times{\cal K} -matrix {\cal S} whose ({\bf k},{\bf k}) th entry equals 1/{|{\bf k}|\choose k_{1},\ldots,k_{J}} . This Tikhonov regularization guarantees invertibility of {\cal M}+{\cal S} as {\cal M} is symmetric and positive semi-definite. 3 General Asymptotic Upper Bound The error of the estimator ( 2.7 ) is decomposed as follows \hat{g}(x)-g(x)\,=\,{\cal B}_{1}\,+\,{\cal B}_{2}\,+\,{\cal B}_{3}\,+\,{\cal V}\,, (3.1) where \displaystyle{\cal B}_{1} \displaystyle\,:=\,-{\bf e}_{0}^{\dagger}({\cal M}+{\cal S})^{-1}\,{\cal S}\,{\bf G}\,, \displaystyle{\cal