Automatic reproducing kernel and regularization for learning convolution kernels

Haibo Li, Fei Lu · 2025

Read Automatic reproducing kernel and regularization for learning convolution kernels by Haibo Li, Fei Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11944.txt.

Introduction

Haibo Li and Fei Lu let data choose its own kernel for learning blur rules in a 2025 paper. Many tasks need the smearing rule inside an operator recovered from samples, an inverse puzzle with too little prior knowledge to pick a kernel by hand.

The samples and the operator together grow a tailor made function space, with no manual kernel hunt. For finite data, a short basis list spans all standard estimators, from least squares to smoothed and iterative fits. Scaled algorithms follow. Tests on three operator kinds beat shelf kernels and ridge fits throughout.

Catalog record

CREATORHaibo Li, Fei Lu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-11944

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automatic reproducing kernel and regularization for learning convolution kernels Haibo Li, Fei Lu Abstract Learning convolution kernels in operators from data arises in numerous applications and represents an ill-posed inverse problem of broad interest. With scant prior information, kernel methods offer a natural nonparametric approach with regularization. However, a major challenge is to select a proper reproducing kernel, especially as operators and data vary. We show that the input data and convolution operator themselves induce an automatic, data-adaptive RKHS (DA-RKHS), obviating manual kernel selection. In particular, when the observation data is discrete and finite, there is a finite set of automatic basis functions sufficient to represent the estimators in the DA-RKHS, including the minimal-norm least-squares, Tikhonov, and conjugate-gradient estimators. We develop both Tikhonov and scalable iterative and hybrid algorithms using the automatic basis functions. Numerical experiments on integral, nonlocal, and aggregation operators confirm that our automatic RKHS regularization consistently outperforms standard ridge regression and Gaussian process methods with preselected kernels. 1 Introduction Kernel functions play a fundamental role in defining operators between function spaces, enabling the representation of nonlocal or long-range interactions between variables. Such kernel-based operators permeate diverse fields: they describe nonlocal diffusion and peridynamic mechanics in partial differential equations (PDEs) [ 4 , 41 , 17 , 46 , 16 , 27 , 6 , 10 ] , govern anomalous transport in fractional diffusion and Lévy processes [ 2 , 18 ] , and underpin advanced image-processing techniques [ 5 , 22 , 34 ] . More recently, they have become central in operator-learning frameworks for scientific machine learning, from DeepONets [ 38 ] and Fourier neural operators [ 33 , 29 ] , nonlocal neural networks [ 45 , 1 ] , and kernel methods [ 9 , 39 , 15 ] . Motivated by these applications, a natural and challenging inverse problem arises: given pairs of inputs and outputs, how can one accurately recover the underlying kernel? We address this question in the linear setting, where the operator acts by convolution against a functional of the input function. By framing kernel recovery as a deconvolution inverse problem, we lay the groundwork for rigorous analysis and practical algorithms that learn these kernels directly from data, bridging the gap between classical inverse problems and modern data-driven operator learning. 1.1 Problem statement We study the problem of estimating a convolution kernel \phi:\mathcal{S}=[0,1]\to\mathbb{R} in the operator R_{\phi}:\mathbb{X}\to\mathbb{Y} of the form R_{\phi}[u](x)=\int_{\mathcal{S}}\phi(s)\,g[u](x,s)\,ds,\quad x\in\mathcal{X}=\{x_{j}\}_{j=1}^{J}\subset[0,1], (1.1) based on discrete and noisy input-output pairs \mathcal{D}=\{(u_{k}(y_{i}),f_{k}(x_{j})):\;1\leq k\leq n_{0},\;1\leq i\leq 3J,\;1\leq j\leq J\}. (1.2) Here, \{y_{i}\}_{i=1}^{3J} and \{x_{j}\}_{j=1}^{J} are uniform meshes of [-1,2] and [0,1] with mesh sizes \Delta x=y_{i+1}-y_{i}=x_{j+1}-x_{j}=\frac{1}{J} , and these data are generated according to f_{k}(x_{j})=R_{\phi}[u_{k}](x_{j})+\epsilon_{k}(x_{j}),\quad\epsilon_{k}(x_{j})\stackrel{{\scriptstyle\text{i.i.d.}}}{{\sim}}\mathcal{N}(0,\sigma^{2}/\Delta x). The input function space \mathbb{X}\subset L^{2}([-1,2]) and the functional g[u](x,s) are problem-specific (see Examples 1.1 – 1.3 ). Note that R_{\phi} can be a nonlinear functional of u , but it depends linearly on \phi . In this study, we consider a fixed discrete observation set \mathcal{X} and set the output function space to be \mathbb{Y}=L^{2}_{\nu}(\mathcal{X}) with an atomic measure \nu defined by \nu(\{x_{j}\})=1/J . When \mathcal{X} is a continuum set [0,1] with Lebesgue measure, the corresponding output space is L^{2}([0,1]) , the noise is white, and the minimax convergence rates in the sample size n_{0} have been studied in [ 50 ] . Such deconvolution-type problems arise in a wide range of applications, and we present three representative examples. Example 1.1 (Integral operator) Estimate \phi:\mathcal{S}\to\mathbb{R} in the integral operator R_{\phi}[u](x)=\int_{[-1,2]}\phi(x-y)\,u(y)\,dy=\int_{[0,1]}\phi(s)\,u(x-s)\,ds with input space \mathbb{X}=C([-1,2]) . In the form ( 1.1 ), the functional is g[u](x,s)=u(x-s) for (x,s)\in\mathcal{X}\times\mathcal{S} . The input u is a sample of the stochastic process u(y)=\sum_{n=2}^{n_{u}}X_{n}\cos(2\pi ny) with n_{u}\leq+\infty , where the coefficients \{X_{n}\} are independent Gaussian random variables N(0,4\sigma_{n}^{2}) with \sum_{n}n\sigma_{n}<+\infty . Example 1.2 (Nonlocal operator) Estimate \phi:\mathcal{S}\to\mathbb{R} in the nonlocal operator: R_{\phi}[u](x)=\int_{|x^{\prime}|\leq 1}\phi(|x^{\prime}|)\bigl(u(x+x^{\prime})-u(x)\bigr)\,\nu(dx^{\prime})=\int_{[0,1]}\phi(s)\,g[u](x,s)\,ds, with input space \mathbb{X}=C^{1}([-1,2]) and g[u](x,s)=u(x+s)+u(x-s)-2u(x) . This operator arises in peridynamics [ 35 , 46 , 47 ] and the Fokker-Planck equation of Lévy processes [ 2 ] . Example 1.3 (Aggregation operator) Consider the aggregation operator R_{\phi}[u]=\nabla\cdot(u\nabla\Phi*u) in the mean-field equation \partial_{t}u=\nu\Delta u+\nabla\cdot(u\nabla\Phi*u) for interacting particle systems [ 6 , 30 ] . Let \Phi be a radial potential supported on \mathcal{S} and set \phi=\Phi^{\prime} . For u\in\mathbb{X}=C^{1}([-1,2]) , one has R_{\phi}[u](x)=\int_{\mathbb{R}}\phi(|x^{\prime}|)\frac{x^{\prime}}{|x^{\prime}|}\,\partial_{x}\bigl[u(x-x^{\prime})u(x)\bigr]\,\,\nu(dx^{\prime})=\int_{\mathcal{S}}\phi(s)g[u](x,s)\,ds with g[u](x,s)=\partial_{x}[u(x-s)u(x)]-\partial_{x}[u(x+s)u(x)] . We consider input functions u to be random probability density functions u(x)=1+\sum_{n=1}^{n_{u}}\sigma_{n}\,\zeta_{n}\,\cos\bigl(2\pi n\,x\bigr), where \{\zeta_{n}\}_{n\geq 1} are i.i.d. random signs (i.e., \mathbb{P}(\zeta_{n}=\pm 1)=\tfrac{1}{2} ) , and \sigma_{n}>0 with \sum_{n}n\sigma_{n}<1 . 1.2 Main results: automatic reproducing kernel and regularization Challenges in learning kernels. Learning convolution kernels from discrete, noisy observations is a severely ill-posed inverse problem: even small data perturbations can induce large estimation errors, making regularization indispensable. Moreover, with scant prior knowledge of the true kernel, a nonparametric framework is necessary, rendering the choice of regularization norm both critical and nontrivial. Kernel methods are particularly suitable for such inverse problems, as they can non-parametrically approximate the unknown functions with regularization using reproducing kernel Hilbert spaces (RKHS). Hence, they have been widely used in machine learning and inverse problems, dating back from solving the Fredholm equations in [ 43 , 42 ] and functional linear regression [ 44 , 48 ] to the recent studies on learning dynamical systems [ 15 , 21 ] , one-shot stochastic differential equations [ 14 ] , linear responses estimations [ 49 ] , and solvers for nonlinear PDEs [ 9 ] and PDEs on manifolds [ 26 ] , to name just a few. In particular, the representer theorem reduces the problem with finite data to a finite-dimensional form, enabling efficient computation and feature extraction. However, a major obstacle in kernel methods is the choice of the reproducing kernel. Standard options, such as Gaussian or Matérn kernels, come with hyperparameters (e.g., bandwidth or smoothness order) that must be carefully tuned. This process is not only computationally expensive but also fails to exploit the specific structure of the inverse problem at hand. In particular, when learning kernels in operators, the variational normal operator may be rank-deficient or possess zero eigenvalues, rendering conventional kernel selection and hyperparameter tuning virtually intractable. Main results. To overcome this obstacle, we propose an automatic reproducing kernel that is defined directly in terms of the data and the forward operator. By incorporating the normal operator from the variational formulation, our kernel automatically adapts to the geometry and spectral properties of the inverse problem. The resulting data-adaptive (DA) RKHS has a closure that is the space in which we can identify the true convolution kernel. Moreover, we use the representer theorem to derive a set of automatic basis functions that are adaptive to the finite discrete observations and are sufficient to represent the estimators in the DA-RKHS, including the minimal-norm least-squares, Tikhonov, and conjugate-gradient estimators. These basis functions make mesh-free regression possible and reveal the finite-dimensional nature of the seemingly infinite-dimensional inverse problem of deconvolution. Building on this theory, we develop two families of regularization algorithms for efficient implementations of the automatic reproducing kernel: • Tikhonov methods based on matrix decomposition for small to medium datasets, with regularization parameters chosen via the L-curve or generalized cross-validation criteria. • Iterative and hybrid regularization schemes that rely solely on matrix-vector products, which are scalable

Read the complete text