Asymptotic Equivalence for Nonparametric Generalized Linear Models
Ion Grama, Michael Nussbaum Institute of Mathematics, Chişinău, Moldova and Weierstrass Institute, Berlin, Germany · 2024
Read Asymptotic Equivalence for Nonparametric Generalized Linear Models by Ion Grama, Michael Nussbaum Institute of Mathematics, Chişinău, Moldova and Weierstrass Institute, Berlin, Germany (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-15057.txt.
Introduction
Grama and Nussbaum prove that a broad class of regression problems with non-Gaussian noise behaves, for large samples, just like the same problem with Gaussian noise. The closeness is measured by Le Cam's deficiency distance, which means the two settings are equally hard for any decision with bounded loss. Each sample comes from the same exponential family, but the samples are not identical copies.
The result needs the regression curve to sit in a Hölder ball with smoothness above one half. The proof builds on a functional version of the Hungarian construction for partial sums. The work extends a recent Gaussian approximation for the simpler identical-sample model.
Catalog record
| CREATOR | Ion Grama, Michael Nussbaum Institute of Mathematics, Chişinău, Moldova and Weierstrass Institute, Berlin, Germany |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 109 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-15057
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Asymptotic Equivalence for Nonparametric Generalized Linear Models Ion Grama, Michael Nussbaum Institute of Mathematics, Chişinău, Moldova and Weierstrass Institute, Berlin, Germany Abstract We establish that a non-Gaussian nonparametric regression model is asymptotically equivalent to a regression model with Gaussian noise. The approximation is in the sense of Le Cam’s deficiency distance Δ \Delta ; the models are then asymptotically equivalent for all purposes of statistical decision with bounded loss. Our result concerns a sequence of independent but not identically distributed observations with each distribution in the same real-indexed exponential family. The canonical parameter is a value f ( t i ) f(t_{i}) of a regression function f f at a grid point t i t_{i} (nonparametric GLM). When f f is in a Hölder ball with exponent β > 1 2 , \beta>\frac{1}{2}, we establish global asymptotic equivalence to observations of a signal Γ ( f ( t ) ) \Gamma(f(t)) in Gaussian white noise, where Γ \Gamma is related to a variance stabilizing transformation in the exponential family. The result is a regression analog of the recently established Gaussian approximation for the i.i.d. model. The proof is based on a functional version of the Hungarian construction for the partial sum process. 1 Research supported by the Deutsche Akademische Austauschdienst and by the Deutsche Forschungsgemeinschaft. 1990 Mathematics Subject Classification. Primary 62B15; Secondary 62G07, 62G20 Key words and phrases. Nonparametric regression, statistical experiment, deficiency distance, global white noise approximation, exponential family, variance stabilizing transformation. 1 Introduction The remarkable success of the Le Cam’s asymptotic theory is mostly due to the power of the concept of weak convergence of statistical experiments, which can be established via LAN conditions. Weak convergence takes place for experiments localized at the normalizer rate for the underlying central limit theorem, i. e. the usual n^{-1/2}. However this is useless if estimators have a slower rate of convergence, which happens with nonparametric estimation problems of the ”ill posed” type. It is therefore natural to abandon the localization concept in this case and to replace limits of experiments by approximations in the sense of Le Cam’s deficiency pseudodistance \Delta. The \Delta -distance can be accessed via coupling of likelihood processes and new results on strong approximation for sums of random variables. We refer to Koltchinskii [ 22 ] for a result on empirical processes in the i. i. d. case. The global \Delta -distance for nonparametric Gaussian experiments was first studied by Brown and Low [ 5 ] , who showed that a normal nonparametric regression is asymptotically equivalent to its continuous version - the signal recovery problem in Gaussian white noise. Then in Nussbaum [ 31 ] it was established that density estimation from i. i. d. data on an interval is asymptotically equivalent to the signal recovery problem, where the signal is the root density. The two sequences of experiments are then accompanying in the sense that their deficiency pseudodistance \Delta tends to zero. This can be regarded as the natural generalization of the classical local asymptotic normality theory to ”ill posed” problems. The implication for decision theory is ”automatic” transfer of risk bounds from one sequence to another. The purpose of the present paper is to accomplish a logical next step in these developments, i. e. to treat the case of non-Gaussian nonparametric regression. Our model is such that at points t_{i}=i/n, i=1,...,n, we observe independent r.v.’s X_{i}, which follow a distribution from an exponential family {\cal P} with parameters \theta_{i}=f(t_{i})\in\Theta, where f:[0,1]\rightarrow\Theta is an unknown function to be estimated. The function f is assumed to belong to the smoothness class \Sigma. The main result of the paper is asymptotic equivalence of this model to a Gaussian experiment of the homoscedastic form dY_{t}^{n}=\Gamma\left(f(t)\right)dt+\frac{1}{\sqrt{n}}dW_{t},\quad t\in[0,1], (1.1) with f\in\Sigma, where \Gamma(\theta):\Theta\rightarrow R is a function such that \Gamma^{\prime}(\theta)=I(\theta)^{1/2} and I(\theta) is the Fisher information in the local exponential family {\cal P} (see Sections 2.2 and 3.3 for a relation with the variance-stabilizing transformation pertaining to {\cal P} ). Here W is the standard Wiener process, and f runs a set of functions in a Hölder ball with exponent \beta>\frac{1}{2} . Note that our function f is tied to the canonical parametrization of the exponential family, while the ”natural” parameter (the intensity for the Poisson case etc.) is generally different. But there is a smooth parameter transformation \lambda=b(\theta) (defined in Section 2.2 below) which permits to formulate global results of the type ( 1.1 ) in ”natural” regression models. Some examples are: [1] Poisson case: X_{i} is Poisson [g(t_{i})], where g is a function on [0,1] in a Hölder ball with exponent \beta>\frac{1}{2}, with values in [\epsilon,\epsilon^{-1}], for some \epsilon>0 . The Poisson intensity is \lambda=b(\theta), where b is a strictly increasing smooth function. Defining the function F(\lambda)=\Gamma(b^{-1}(\lambda)), we obtain (see Section 4 below) F(\lambda)=2\sqrt{\lambda} . The accompanying Gaussian experiment is dY_{t}^{n}=2\sqrt{g(t)}\,dt+\frac{1}{\sqrt{n}}dW_{t},\quad t\in[0,1]. (1.2) [2] Bernoulli case: X_{i} is Binomial [1,g(t_{i})], g as above, but with values in [\epsilon,1-\epsilon], for some \epsilon>0 . The natural parameter is \lambda=b(\theta), for some function b with properties as above. We have F(\lambda)=2\arcsin(\sqrt{\lambda}), and the accompanying Gaussian experiment is dY_{t}^{n}=2\arcsin\sqrt{g(t)}\,dt+\frac{1}{\sqrt{n}}dW_{t},\quad t\in[0,1]. [3] Gaussian variance case: X_{i} is N[0,g(t_{i})], g as in example 1. We have F(\lambda)=2^{-1/2}\log\lambda, and dY_{t}^{n}=\frac{1}{\sqrt{2}}\log g(t)\,dt+\frac{1}{\sqrt{n}}dW_{t},\quad t\in[0,1]. (1.3) For more details see Section 4 . We chose to give here the continuous versions of the accompanying experiments, but the discrete versions are also available. The motivation for this paper can be concisely expressed in three points. [A] The first example essentially recovers the result of [ 31 ] on i.i.d. data with density f on the unit interval, since the proof in [ 31 ] used a Poisson approximation. In the second example, F(\lambda)=2\arcsin(\sqrt{\lambda}) is the well known variance stabilizing transformation for the binomial distribution. For scale parameters like the Gaussian variance, the logarithm is also known to be a stabilizing transformation, and the same applies to 2\lambda^{1/2} in the Poisson case. Thus, in the evolving theory of asymptotic equivalence of experiments, we have achieved a better understanding of where global closed form approximations like ( 1.1 ) arise from. The formal connection to differential geometric theory in statistics seems very interesting and remains to be explored (cp.Čencov [ 8 ] , Amari et al. [ 1 ] ). [B] The case when the function f is in a linear parametric class \left\{f(x)=\beta x,\;\beta\in R\right\} is known as a generalized linear model (GLM). The inverse of the transformation b(\theta) would then be the canonical link function. Accordingly, our model is of the type nonparametric GLM , cp. Green and Silverman [ 19 ] , sec. 5.1.2. Models like these, which offer tremendous flexibility, have received much attention in the recent literature, see also Fan and Gijbels [ 14 ] . It would be beyond the scope of this paper to treat the many (semiparametric) variants and extensions; we refer to [ 19 ] and [ 14 ] . In particular we do not discuss logit and probit analysis in our context (cp. example 3, p. 92 in [ 19 ] ). Empirical process theory has also been applied by Mammen and van de Geer [ 28 ] in our model (in a more general variant), for constructing