Power-boosting in Specification Tests using Kernel Directional Component
Rui Cui, Yuhao Li, Xiaojun Song · 2025
Read Power-boosting in Specification Tests using Kernel Directional Component by Rui Cui, Yuhao Li, Xiaojun Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04900.txt.
Introduction
Rui Cui, Yuhao Li, and Xiaojun Song study why some statistical tests lose power when samples are small, in work from 2025. They focus on kernel based tests that check whether a model fits data well. In practice, such tests often work less well than large sample theory suggests.
The authors trace the problem to error in the parts of the test that point in certain directions. Parts tied to small values are unstable and add noise more than signal. Cutting those parts away can help. Their account explains why useful power often sits in a small, low part of the test.
Catalog record
| CREATOR | Rui Cui, Yuhao Li, Xiaojun Song |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 96 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04900
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Power-boosting in Specification Tests using Kernel Directional Component Thanks: Li acknowledges support from the Research Development Fund (RDF-23-02-022) of the Xi’an Jiaotong-Liverpool University. Song acknowledges support from the National Natural Science Foundation of China (Grant Numbers 72373007 and 72333001). Rui Cui, Yuhao Li, Xiaojun Song Abstract We propose power-boosting strategies for kernel-based specification tests in conditional moment models, with a focus on the Kernel Conditional Moment (KCM) test. By decomposing the KCM statistic into spectral components, we demonstrate that truncating poorly estimated directions and selecting kernels based on a non-asymptotic signal-to-noise ratio significantly improves both test power and size control. Our theoretical and simulation results demonstrate that, while divergent component weights may offer higher asymptotic power, convergent component weights perform better in finite samples. The methods outperform existing tests across various settings and are illustrated in an empirical application. Keywords: Berry–Esseen inequality; Convergence of eigenspaces; Kernel principal component analysis; Reproducing kernel Hilbert space 1 Introduction We consider the following general parametric regression model: Y=\mathcal{M}_{\theta_{0}}(X)+\varepsilon_{\theta_{0}}, where \mathcal{M}_{\theta_{0}}(X) is a parametric function indexed by the unknown parameter vector \theta_{0}\in\Theta\subset\mathbb{R}^{q} , X\in\mathcal{X}\subset\mathbb{R}^{q} is the covariate, Y\in\mathcal{Y}\subset\mathbb{R} is the response, and \varepsilon_{\theta_{0}}=\varepsilon_{0} is the model error. This setup covers a wide range of models, including linear and nonlinear regression, quantile regression, treatment effect models, and instrumental variables models. Correct specification of \mathcal{M}_{\theta_{0}} is essential for valid inference. Our goal is to test the following hypotheses: H_{0}:\mathbb{P}\left(\mathbb{E}[\varepsilon_{0}\mid X]=0\right)=1\text{ for some }\theta_{0}\in\Theta versus H_{1}:\mathbb{P}\left(\mathbb{E}[\varepsilon_{\theta}\mid X]\neq 0\right)>0\text{ for all }\theta\in\Theta. The Integrated Conditional Moment (ICM) test of Bierens 1982 is a classical approach for assessing model specification. The ICM test statistic is given by n\hat{T}_{ICM}=\frac{1}{n}\sum_{i,j=1}^{n}\varepsilon_{\hat{\theta},i}\exp\left(-\frac{\lVert x_{i}-x_{j}\rVert_{2}^{2}}{2}\right)\varepsilon_{\hat{\theta},j}, where \varepsilon_{\hat{\theta},i}=y_{i}-\mathcal{M}_{\hat{\theta}}(x_{i}) , \hat{\theta} is a consistent estimator of \theta_{0} , and n is the sample size. Here, \lVert\cdot\rVert_{2} denotes the Euclidean norm. Muandet et al. 2020 generalized the ICM test to the Kernel Conditional Moment (KCM) test by allowing any integrally strictly positive definite (ISPD) reproducing kernel k(x_{i},x_{j}) : n\hat{T}_{KCM}=\frac{1}{n}\sum_{i,j=1}^{n}\varepsilon_{\hat{\theta},i}k(x_{i},x_{j})\varepsilon_{\hat{\theta},j}. Escanciano 2024 further developed related test statistics using a Gaussian Process perspective, with particular attention to handling the estimation effect from \hat{\theta} . KCM-type test statistics have gained significant attention in the literature due to their omnibus property—that is, their ability to detect a wide range of deviations from the null hypothesis—as well as their relative insensitivity to the dimensionality of the covariates. All KCM-type test statistics are designed to estimate the quantity \mathbb{E}[\varepsilon_{0}\,k(X,X^{\prime})\,\varepsilon_{0}^{\prime}] where (X^{\prime},\varepsilon_{0}^{\prime}) is an independent copy of (X,\varepsilon_{0}) . The null hypothesis holds if and only if this quantity is zero; see Muandet et al. 2020 , Escanciano 2024 . A key observation underlying our analysis is that, by Mercer’s theorem, the kernel k admits the spectral decomposition k(x,y)=\sum_{i=1}^{\infty}\mu_{i}\,\phi_{i}(x)\,\phi_{i}(y), where \{\mu_{i}\}_{i=1}^{\infty} are the eigenvalues and \{\phi_{i}\}_{i=1}^{\infty} are the corresponding eigenfunctions of the integral operator L_{k} defined by L_{k}f(x)=\int k(x,y)\,f(y)\,d\mathbb{P}(y),\quad\forall f\in L^{2}(\mathbb{P}),\ x\in\mathcal{X}, with L^{2}(\mathbb{P}) denoting the space of square-integrable functions with respect to \mathbb{P} . The eigenfunctions satisfy L_{k}\phi_{i}(x)=\mu_{i}\,\phi_{i}(x),\quad\forall x\in\mathcal{X},\ i=1,2,\ldots, and the collection \{\phi_{i}\}_{i=1}^{\infty} forms a complete orthonormal basis for L^{2}(\mathbb{P}) . Substituting the spectral decomposition into the unconditional moment yields \mathbb{E}[\varepsilon_{0}\,k(X,X^{\prime})\,\varepsilon_{0}^{\prime}]=\sum_{i=1}^{\infty}\mu_{i}\,\left(\mathbb{E}[\varepsilon_{0}\,\phi_{i}(X)]\right)^{2}. In this paper, we will call quantities \{\mathbb{E}[\varepsilon_{0}\,\phi_{i}(X)]\}_{i=1}^{\infty} directional components. The representation above reveals that the null hypothesis holds if and only if all the directional components \mathbb{E}[\varepsilon_{0}\,\phi_{i}(X)] are zero for every i\geq 1 . To formally establish this connection, we introduce the following conditions: Assumption A. (i) \mathbb{E}(\varepsilon_{\theta}|X)\in L^{2}(\mathbb{{P}}) , furthermore, there exists a unique \theta_{0}\in\Theta for which \mathbb{E}(\varepsilon_{\theta_{0}}|X)=0 a.s., and \mathbb{P}(\mathbb{E}(\varepsilon_{\theta}|X)=0)<1 for all \theta\in\Theta,\theta\neq\theta_{0} ; (ii) The reproducing kernel is integrally strict positive definite (ISPD) and bounded in the sense that \sup_{x\in\mathcal{X}}k(x,x)<\infty ; (iii) |\mathbb{E}(\varepsilon_{0}\phi_{i}(X))|<M,\forall i\geq 1 . Condition (i) ensures that \mathbb{E}(\varepsilon_{\theta}|X) is a well-defined function and is globally identified. This condition holds in most models considered in the literature Hall 2003 . Condition (ii) requires that the reproducing kernel k is integrally strictly positive definite (ISPD), meaning that \int_{\mathcal{X}}\int_{\mathcal{X}}k(x,y)f(x)f(y)\,\mathrm{d}\mathbb{P}(x)\mathrm{d}\mathbb{P}(y)>0,\quad\forall f\in L^{2}(\mathbb{P}),\ f\neq 0. This ISPD property ensures that, under the alternative hypothesis, there exists at least one directional component \mathbb{E}[\varepsilon_{0}\phi_{i}(X)]\neq 0 for some i\geq 1 . A typical example of an ISPD kernel is the Gaussian kernel k(x,y)=\exp(-\gamma\|x-y\|^{2}) for some \gamma>0 . Condition (iii) imposes a mild regularity requirement on the residuals, ensuring that the projections onto the eigenfunctions are uniformly bounded: |\mathbb{E}(\varepsilon_{0}\phi_{i}(X))|<M for all i\geq 1 . This leads to the following theorem: Theorem 1 Under Assumption A, the null hypothesis H_{0} is equivalent to the condition that the directional components \mathbb{E}[\varepsilon_{0}\phi_{i}(X)] are zero for all i\geq 1 . Proof. See Online Appendix A.1 . Escanciano 2024 points out that the KCM test statistic may exhibit low power when deviations from the null hypothesis occur in directions corresponding to small eigenvalues, since the KCM statistic inherently downweights these components. This raises a natural question: can the power of the KCM test be improved by adjusting the weights assigned to the directional components? In this paper, we address this question by developing estimation methods for the directional components and systematically evaluating the testing performance under various weighting schemes. Our main findings are as follows: (i) For a fixed kernel, simply increasing the weights on directional components associated with small eigenvalues does not substantially improve power; in such cases, increasing the sample size remains the most effective strategy. (ii) For a given sample size, however, power can be significantly enhanced by selecting a kernel that increases the detectability of directional components associated with larger eigenvalues. The first finding can be intuitively explained as follows: estimating each directional component requires accurately estimating the corresponding eigenfunction \phi_{i} associated with the i -th largest eigenvalue \mu_{i} . However, the quality of eigenfunction estimation is not uniform across all indices i . In particular, eigenfunctions associated with smaller eigenvalues (i.e., higher-frequency components) are estimated with greater error and require larger sample sizes for a reliable approximation. As a result, directional components associated with these eigenfunctions are inherently more challenging to estimate accurately. Therefore, simply increasing the weights on components tied to small eigenvalues does not meaningfully improve test power. This estimation quality argument motivates a power-boosting strategy based on truncating, or using a low-rank approximation of, the kernel matrix \boldsymbol{K}_{\gamma} , where \gamma is a kernel tuning parameter (such as the bandwidth in the Gaussian kernel k_{\gamma}(x,y)=\exp(-\gamma\|x-y\|^{2}) ). Specifically, we retain only the leading eigenvalues and corresponding eigenvectors, thereby focusing on the most reliably estimated components. To further enhance power, we recommend tuning the kernel parameter \gamma to maximize the detectability of directional components associated with larger eigenvalues. This can be achieved by studentizing the estimator of the weighted sum \sum_{i=1}^{J}\mu_{i}\left(\mathbb{E}[\varepsilon_{0}\phi_{i}(X)]\right)^{2} , where J is the truncation level, and applying the Berry–Esseen inequality to ensure that the studentized statistic is well-approximated by a standard normal distribution in finite samples. Selecting \gamma to maximize this studentized statistic yields substantial power gains by facilitating optimal kernel selection. Motivated by the directional component framework, we propose a family of test statistics of the form n\hat{T}=\sum_{i=1}^{J}\omega_{i}\left(\boldsymbol{\varepsilon_{0}}^{\top}\boldsymbol{u}_{i}\right)^{2}, where \boldsymbol{u}_{i} denotes the eigenvector of the kernel matrix \boldsymbol{K} corresponding to the i -th largest eigenvalue, and \boldsymbol{\varepsilon_{0}} is the vector of model errors. The choice of weights \{\omega_{i}\}_{i=1}^{\infty} plays a critical role in determining the statistical properties of the test. In particular, using divergent weights (i.e., \sum_{i=1}^{\infty}\omega_{i}\to\infty ) can, in