PCA-Guided Quantile Sampling: Preserving Data Structure in Large-Scale Subsampling
Foo Hui-Mean, Yuan-chin Ivan Chang · 2025
Read PCA-Guided Quantile Sampling: Preserving Data Structure in Large-Scale Subsampling by Foo Hui-Mean, Yuan-chin Ivan Chang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-18249.txt.
Introduction
Foo Hui-Mean and Yuan-chin Ivan Chang introduce a sampling method that shrinks huge datasets while keeping their shape. Analysts often subsample big data for speed, but careless sampling distorts the very patterns under study. Their method, PCA-guided quantile sampling, keeps the original features untouched.
Principal components, the data's main axes of variation, guide a quantile-based stratification of the sample. Theory guarantees how closely the sample's distribution tracks the full data, measured by divergence and distance. Practical rules follow for choosing components, bins, and sampling rates. Tests on synthetic and real data check the fidelity. Honest subsamples make fast analysis safer for high-stakes conclusions.
Catalog record
| CREATOR | Foo Hui-Mean, Yuan-chin Ivan Chang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-18249
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PCA-Guided Quantile Sampling: Preserving Data Structure in Large-Scale Subsampling Foo Hui-Mean, Yuan-chin Ivan Chang Abstract We introduce Principal Component Analysis-guided Quantile Sampling (PCA-QS), a novel sampling framework designed to preserve both the statistical and geometric structure of large-scale datasets. Unlike conventional PCA, which reduces dimensionality at the cost of interpretability, PCA-QS retains the original feature space while using leading principal components solely to guide a quantile-based stratification scheme. This principled design ensures that sampling remains representative without distorting the underlying data semantics. We establish rigorous theoretical guarantees, deriving convergence rates for empirical quantiles, Kullback–Leibler divergence, and Wasserstein distance, thus quantifying the distributional fidelity of PCA-QS samples. Practical guidelines for selecting the number of principal components, quantile bins, and sampling rates are provided based on these results. Extensive empirical studies on both synthetic and real-world datasets show that PCA-QS consistently outperforms simple random sampling, yielding better structure preservation and improved downstream model performance. Together, these contributions position PCA-QS as a scalable, interpretable, and theoretically grounded solution for efficient data summarization in modern machine learning workflows. Keywords and Phrases: PCA-guided Quantile Sampling; structure-preserving sampling; quantile stratification; convergence rates; large-scale data reduction. 1 Introduction The exponential growth of modern datasets poses a fundamental challenge: how to reduce data volume while preserving both statistical fidelity and structural integrity. Conventional sampling strategies, although effective for data reduction, often compromise representativeness, distort key distributions, and degrade the performance of downstream models. To address these limitations, we propose PCA-QS, a principled framework for subsampling that preserves the intrinsic structure of complex datasets. In contrast to conventional PCA, which projects data into a lower-dimensional subspace—often sacrificing interpretability—PCA-QS retains the full original feature space. Instead, it leverages the leading principal components purely as structural guides to define a quantile-based stratification scheme. Sampling is then conducted within these quantile strata, ensuring that the selected subset faithfully captures major variance patterns, distributional characteristics, and the underlying geometric structure of the data. The proposed method is supported by rigorous theoretical analysis. By building on established results for empirical quantiles, Kullback–Leibler (KL) divergence, and Wasserstein distance, we demonstrate that PCA-QS exhibits well-defined convergence behavior: as the sample size grows, the distribution of the sampled subset converges uniformly to that of the full dataset. These guarantees provide a robust theoretical foundation for the method’s representativeness and structural preservation. PCA-QS makes several notable contributions: First, it introduces a novel sampling paradigm that combines the interpretability of the original feature space with the structural insight provided by principal components. Second, it offers a theoretical framework for parameter selection, including the number of principal components and quantile bins, grounded in explicit convergence rates. Third, it demonstrates empirically—across synthetic and real-world datasets—that PCA-QS consistently outperforms standard simple random sampling (SRS) in maintaining structural fidelity and improving the performance of downstream machine learning models. Finally, it provides a scalable, interpretable, and theoretically justified solution for efficient data summarization in modern machine learning workflows. Subsampling and data summarization are core strategies for scaling statistical and machine learning methods to large datasets. Classic techniques include simple random sampling (SRS) and stratified sampling, which aim to maintain representativeness by leveraging known strata such as class labels or demographic groups Owen 2013 . More recently, structure-aware methods such as coreset construction and matrix sketching have emerged to provide data compression with provable error bounds for downstream tasks like regression, clustering, and kernel estimation Feldman et al. 2011 ; Phillips 2016 ; Drineas et al. 2006 . Principal Component Analysis (PCA) remains one of the most widely used tools for revealing latent structure, and numerous variants have been developed to enhance its robustness or enforce sparsity Zou et al. 2006 ; Candès et al. 2011 . However, such methods primarily focus on transforming the feature space rather than selecting a representative subset of data points. Existing importance sampling and Markov chain Monte Carlo techniques address computational scalability for inference and optimization but generally lack guarantees on preserving both distributional and geometric properties in the subsample Bardenet et al. 2017 ; Ma et al. 2015 . By contrast, PCA-QS combines the strengths of PCA for discovering major variance directions with a principled quantile-based stratification scheme. This design retains the original feature space while ensuring that the selected subset approximates the full data distribution with provable rates under both Kullback–Leibler divergence and Wasserstein distance Peyré and Cuturi 2019 ; Panaretos and Zemel 2020 . Therefore, PCA-QS bridges the gap between classical stratification and modern coreset techniques, offering a flexible and interpretable solution for large-scale data summarization in machine learning workflows. The remainder of this paper is organized as follows: Section 2 reviews related work and details the PCA-QS methodology. Section 3 presents empirical evaluations on synthetic and real-world datasets. Section 4 discusses broader implications and concludes the study. 2 Method Principal Component Analysis (PCA) is a classical technique for dimensionality reduction that projects high-dimensional data onto a lower-dimensional subspace that retains the maximal variance. Given a dataset \mathbf{X}\in\mathbb{R}^{n\times p} , the PCA transformation is typically obtained via the Singular Value Decomposition (SVD): \mathbf{X}=\mathbf{U}\mathbf{D}\mathbf{V}^{\top}, where \mathbf{U}\in\mathbb{R}^{n\times p} contains the left singular vectors, \mathbf{V}\in\mathbb{R}^{p\times p} the right singular vectors, and \mathbf{D}\in\mathbb{R}^{p\times p} is a diagonal matrix of singular values. The projection of the dataset onto the top k principal components yields the reduced representation: \mathbf{Z}=\mathbf{U}_{k}\mathbf{D}_{k}, where \mathbf{U}_{k} and \mathbf{D}_{k} contain the first k columns and leading singular values, respectively. While PCA is effective in capturing dominant variance directions, it does not inherently reduce the number of data points. To address this, the PCA-QS framework augments PCA with a stratified sampling mechanism to achieve both data summarization and structural fidelity. Property of Quantiles: Let X_{1},X_{2},\dots,X_{n} be an i.i.d. sample from a distribution with cumulative distribution function (CDF) F(x) . The empirical CDF is defined as F_{n}(x)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}(X_{i}\leq x) . For a quantile level p\in(0,1) , the true quantile is q_{p}=F^{-1}(p) , and the empirical quantile is q_{p}^{n}=F_{n}^{-1}(p) . By the Glivenko–Cantelli theorem van der Vaart and Wellner 1996 , the empirical CDF converges uniformly to the true CDF as n\to\infty , i.e., \sup_{x}|F_{n}(x)-F(x)|=O(n^{-1/2}) . Consequently, the quantile estimation error satisfies q_{p}^{n}-q_{p}=O(n^{-1/2}) . 2.1 PCA-Guided Quantile Sampling Method Sampling in Principal Component Space: PCA-QS stratifies data along principal component directions using quantile partitions to facilitate representative sampling. For the case of a single principal component ( \text{num.\ of PCs}=1 ) and g quantile groups, the data is partitioned into intervals as follows: [\text{min},Q_{1}),[Q_{1},Q_{2}),\ldots,[Q_{g-1},\text{max}], (1) ensuring that each group contains an approximately equal proportion of the data. Quantile Assignment for Multiple Components In the general case with multiple principal components ( \text{num.\ of PCs}>1 ), quantile stratification extends across a multi-dimensional subspace. For each principal component z_{j} , the data is divided into quantile bins: \text{Quantile Groups for Component }j:[\text{Min},Q_{1}),[Q_{1},Q_{2}),\ldots,[Q_{g-1},\text{Max}]. Composite quantile groups are then formed by combining the bin assignments across the selected components. These composite groups are uniquely encoded to facilitate balanced sampling in the reduced PCA space. Sampling within each composite quantile group is typically random to promote unbiased representation. The number of retained samples per group is defined as: \text{Sample Size per Group}=\min\left(\left\lceil\text{retention rate}\times N_{g}\right\rceil,N_{g}\right), where N_{g} is the size of the group, and the retention rate controls the overall proportion of retained data. This process preserves both