Covariance test for discretely observed functional data: when and how it works
Yang Zhou · 2025
Read Covariance test for discretely observed functional data: when and how it works by Yang Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04962.txt.
Introduction
Yang Zhou and co-authors tackled a testing problem in curve data in two thousand twenty-five. Many studies record a whole curve for each person, like growth over time. Tests often compare how curves spread around their average. Past tests assumed full, clean curves were observed. Real data instead give scattered points with noise for each person.
The team built a test that first smooths the scattered points. It then uses the main shapes found in the pooled curves. Their math shows the test stays valid as more shapes are added. With dense enough sampling, it acts as if full curves were seen. Tests of this kind support work in health, growth, and motion studies. The paper gives clear rules for when the method can be trusted.
Catalog record
| CREATOR | Yang Zhou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04962
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Covariance test for discretely observed functional data: when and how it works? Thanks: Corresponding author: [email protected]. Fang Yao’s research is partially supported by the National Key R&D Program of China (No. 2022YFA1003800), the National Natural Science Foundation of China (No. 12292981, 11931001), the LMAM, the Fundamental Research Funds for the Central Universities and the LMEQF. Yang Zhou’s research is partially supported by the National Natural Science Foundation of China (No. 12401334). Jin Yang’s research is partially supported by NIH Intramural Research Program, USA. Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). Yang Zhou Abstract For covariance test in functional data analysis, existing methods are developed only for fully observed curves, whereas in practice, trajectories are typically observed discretely and with noise. To bridge this gap, we employ a pool-smoothing strategy to construct an FPC-based test statistic, allowing the number of estimated eigenfunctions to grow with the sample size. This yields a consistently nonparametric test, while the challenge arises from the concurrence of diverging truncation and discretized observations. Facilitated by advancing perturbation bounds of estimated eigenfunctions, we establish that the asymptotic null distribution remains valid across permissable truncation levels. Moreover, when the sampling frequency (i.e., the number of measurements per subject) reaches certain magnitude of sample size, the test behaves as if the functions were fully observed. This phase transition phenomenon differs from the well-known result of the pooling mean/covariance estimation, reflecting the elevated difficulty in covariance test due to eigen-decomposition. The numerical studies, including simulations and real data examples, yield favorable performance compared to existing methods. Keywords: Diverging truncation; Perturbation bounds; Phase transition; Functional principal components. 1 Introduction Functional data, referring to data generated from continuous underlying processes, are collected in many scientific fields such as medical imaging, speech recognition, molecule biology and environmental meteorology, etc. Functional data analysis (FDA), a popular branch of statistics over the past decades, has attracted considerable attention, see Ramsay and Silverman 2005 for an introduction and Hsing and Eubank 2015 for some theoretical foundations. The research on hypothesis testing for functional data is mostly concentrated on mean functions ( Fan and Lin 1998 ; Horváth et al. 2013 ; Horváth and Rice 2015 ) with scarce attention paid to covariance operators, which lay the foundation of subsequent analysis such as functional principal component (FPC) and regression analysis ( Yao et al. 2005a ; Yao et al. 2005b ; Hall and Hosseini-Nasab 2006 ; Hall et al. 2006 ; Hall and Horowitz 2007 , among others) , and is the focus of this article. To motivate the proposed testing method, we point out that the challenges to be encountered are two-fold. Functional covariance test is intrinsically a nonparametric test on compact operators that reside in the infinite-dimensional space, and is substantively complicated by discretely observed and noise-contaminated data in practice. For two-sample covariance test, a typical line of existing methods aims to metrizing the difference directly based on appropriate metric/distance (e.g., Hilbert-Schmidt norm), to construct test statistics whose asymptotic distributions depend on infinite-dimensional parameters which are unknown and difficult to estimate in practice. Consequently, these fully-functional tests suffer intensive calculations by some resampling methods (e.g., permutation and/or bootstrap) to obtain approximate critical values (see Pigoli et al. 2014 ; Paparoditis and Sapatinas 2016 ; Boente et al. 2018 , for example) . Another line of research exploited the projection-based methods, defining test statistics on a finite-dimensional space, where the basis is commonly formed by a fixed number of eigenfunctions ( Panaretos et al. 2010 ; Kraus and Panaretos 2012 ; Fremdt et al. 2013 ) . Facilitated by dimension reduction, the derived asymptotic distributions of FPC-based tests are more tractable due to the finite-dimensional approximation, however, the fixed truncation adopted in these works violated the nonparametric nature. It is noteworthy that all methods in literature have been only developed for fully observed functional data that may be viewed as an “ideal” scenario. Recently, Kraus 2019 investigated the k -sample test for covariance operators when functional data are partially observed and established the asymptotic distributions of FPC-based tests for fixed truncations. The “practical” scenario is that one observes n random functions at N_{i} (1\leq i\leq n) discrete time points with noise contamination, where N_{i} may vary for each subject and is often assumed N_{i}\asymp N . Here a_{n}\asymp b_{n} if a_{n}\gtrsim b_{n} and b_{n}\gtrsim a_{n} , where a_{n}\gtrsim b_{n} indicates a_{n}\geq\mathrm{c}b_{n} for some constant \mathrm{c} and large n . There are two typically smoothing strategies when dealing with such discretely observed functional data, one of which is only applicable to sufficiently dense data by pre-smoothing individual curves before further analysis. For sparse functional data, it is suggested to borrow information from all subjects by pooling observations together to estimate population parameters ( Yao et al. 2005a ) . Here the magnitude of N , referred to as the sampling frequency, plays an important role in the choice of smoothing methods and the asymptotic behavior with a phenomenon called “phase transition”. For example, Zhang and Chen 2007 and Kong et al. 2016 have shown that, when N\gtrsim n^{5/4} the mean and covariance estimators based on pre-smoothed curves achieve the parametric \sqrt{n} -convergence. In contrast, the pool-smoothing estimators for the mean and covariance functions attain optimal \sqrt{n} -consistency when N\gtrsim n^{1/4} ( Cai and Yuan 2011 ; Zhang and Wang 2016 ) . These results provide theoretical insights into the advantages of pool-smoothing estimation over the pre-smoothing method. Nevertheless, the above-mentioned phase transition are only valid for mean and covariance estimation, while for rigorous inference, especially involving eigenfunctions based on discretely observed data, there has not been meaningful progress to our knowledge. Viewing ubiquitousness of discretely observed data, we are inspired to bridge the gap from the discretized observations to the continuum and answer the key question: when and how a covariance test procedure works? Despite the possibility of using pre-smoothing or pool-smoothing methods to conduct fully-functional tests, it is challenging to study the theoretical properties due to individual smoothing errors or the complex structure of pool-smoothing estimators, especially when considering a unified framework for functional data from sparse to dense designs. FPC-based tests could effectively capture differences that commonly arise from significant variations across the spectra of covariance operators due to good behaviour of the eigen-system, and their explicit asymptotic distributions could be derived, laying the theoretical foundation for conducting statistical inference. Thus we adopt FPC-based tests in this paper and the number of eigen-projections (i.e., the truncation level) is not fixed but rather allowed to increase with the sample size. This flexible approach places our method within a nonparametric regime, enabling us to enhance statistical power against deviations in any direction with increasing sample size. To bridge the gap between the continuum and discretized observations, a crucial step is to quantify the perturbation bounds for a diverging number of estimated eigenfunctions based on discretely observed data, which is vital for deriving the asymptotic test distribution. A seminal work of Hall and Horowitz 2007 has established sharp bounds for estimated eigenfunctions with diverging index based on the cross-sectional sample covariance that is only