FINDER: Feature Inference on Noisy Datasets using Eigenspace Residuals

Trajan MurphyDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], Akshunna S. DograMathePhysics (India), NSF IAIFI (USA),Department of Mathematics Imperial College London, UK SW7 2AZ [email protected] Hanfeng GuDepartment of Earth and EnvironmentBoston UniversityBos · 2025

Read FINDER: Feature Inference on Noisy Datasets using Eigenspace Residuals by Trajan MurphyDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], Akshunna S. DograMathePhysics (India), NSF IAIFI (USA),Department of Mathematics Imperial College London, UK SW7 2AZ [email protected] Hanfeng GuDepartment of Earth and EnvironmentBoston UniversityBos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19917.txt.

Introduction

Trajan Murphy and five co-authors introduce FINDER, a classification framework built for datasets that are noisy, small, or partly faulty. Instead of treating a dataset as fixed numbers, FINDER views it as one realization of a random field and lifts it into a Hilbert space. A Kosambi-Karhunen-Loève expansion then breaks the data into components whose spectra separate the classes.

The stochastic framing is meant to account honestly for the randomness inside every empirical dataset. On hard, data-poor scientific problems the authors report state-of-the-art results, including staging Alzheimer's disease and spotting deforestation in remote sensing imagery. They close by discussing when FINDER should beat conventional methods, where it fails, and its other limits.

Catalog record

CREATORTrajan MurphyDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], Akshunna S. DograMathePhysics (India), NSF IAIFI (USA),Department of Mathematics Imperial College London, UK SW7 2AZ [email protected] Hanfeng GuDepartment of Earth and EnvironmentBoston UniversityBos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19917

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FINDER: Feature Inference on Noisy Datasets using Eigenspace Residuals Trajan MurphyDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], Akshunna S. DograMathePhysics (India), NSF IAIFI (USA),Department of Mathematics Imperial College London, UK SW7 2AZ [email protected] Hanfeng GuDepartment of Earth and EnvironmentBoston UniversityBoston, MA 02215 [email protected] Caleb Julius MeredithDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected] Mark KonDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], Julio Enrique Castrillón-CandásDepartment of Mathematics and StatisticsBoston UniversityBoston, MA 02215 [email protected], for the Alzheimer’s Disease Neuroimaging Initiative Abstract “Noisy” datasets (regimes with low signal to noise ratios, small sample sizes, faulty data collection, etc) remain a key research frontier for classification methods with both theoretical and practical implications. We introduce FINDER, a rigorous framework for analyzing generic classification problems, with tailored algorithms for noisy datasets. FINDER incorporates fundamental stochastic analysis ideas into the feature learning and inference stages to optimally account for the randomness inherent to all empirical datasets. We construct “stochastic features” by first viewing empirical datasets as realizations from an underlying random field (without assumptions on its exact distribution) and then mapping them to appropriate Hilbert spaces. The Kosambi-Karhunen-Loève expansion (KLE) breaks these stochastic features into computable irreducible components, which allow classification over noisy datasets via an eigen-decomposition: data from different classes resides in distinct regions, identified by analyzing the spectrum of the associated operators. We validate FINDER on several challenging, data-deficient scientific domains, producing state of the art breakthroughs in: (i) Alzheimer’s Disease stage classification, (ii) Remote sensing detection of deforestation. We end with a discussion on when FINDER is expected to outperform existing methods, its failure modes, and other limitations. 1 Introduction Classification problems are of significant interest across a variety of scientific and commercial fields, especially when concerned with “noisy” datasets: settings where the nominal data dimension F \gg than the sample size N , datasets with poor signal to noise ratios, etc. Deep/Machine Learning (ML) methods are particularly known to be susceptible in data-deficient settings LeCun et al. 2014 ; Ng 2004 and thus techniques for performance improvements in these regimes remain of strong interest. We present a multi-faceted novel development within these contexts: a generic and versatile theory for discussing classification problems with applications in treating noisy datasets, validated over a collection of challenging and significant scientific datasets. Broadly speaking, we blend standard feature inference/construction methods with the Kosambi-Karhunen-Loève theorem Loève 1978 from stochastic analysis, by rigorously defining and building “stochastic features” that can help classify the underlying structures while seeing through the inherent “blurriness” of noisy datasets. We begin with binary classification since many classification problems reduce to a set of binary ones. Binary classification involves classifying an input object into one of two classes, \{\mathbf{A},\mathbf{B}\} , \{0,1\} , etc, based on a list of numeric quantities associated with that object. For large F , this becomes computationally intractable as N may be too limited for machine training. Principal component analysis (PCA) was developed largely as a way to effectively reduce the nominal dimension of a given dataset with minimal information loss Kokoszka and Reimherr 2017 . Further developments involved viewing data not as a vector in \mathbb{R}^{F} , but as a function from a closed interval [a,b] to \mathbb{R} , sampled at F points. Such methods for analyzing, constructing machines from, and making predictions based on data comprise the field of functional data analysis (FDA) Kokoszka and Reimherr 2017 ; Horváth and Kokoszka 2012 . FDA is often preferred when F\gg N : even linear discriminant analysis can be outperformed by the much simpler naive Bayesian classification Bickel and Levina 2004 . The Kosambi-Karhunen-Loève expansion (KLE) Schwab and Todor 2006 is a fundamental result in FDA, mildly generalized by us as Thm. 2.1 . It implies that our stochastic features admit a Fourier series like expansion in terms of simpler, computable elements. We pair Thm. 2.1 with novel algorithms for constructing/finding stochastic features with in-built class separability (to the extent such underlying features can exist for the available data). We further prove that digitization does not limit the usefulness of these results and show how optimal choices for truncation may be made. We test our approach on noisy datasets of significant scientific interest, with major improvements on existing state of the art results. FINDER is generic, but geared towards such data-deficient or otherwise noisy settings, its relative performance advantages expectedly increasing with the “noise”. Another advantage lies in the efficiencies it may unlock for otherwise computationally intensive tasks. The inherent robustness to noise and in-built class separability implies nominally intractable datasets can become amenable even to fundamental ML algorithms like support vector machines (SVM) or hidden Markov models (HMM), which can then be used with dramatic improvements in accuracy. Hence, while FINDER comes with additional construction costs of its own, the fact that it can be packaged with simpler methods like SVM provides a pathway to manageable computational costs. FINDER is also relatively robust to unbalanced data: it is a functional analytic schema and the number of available samples per class is a matter of concern only insomuch that the “noisiness” of the class changes with that number. Numerical experiments validating our claims are provided in Sec. 3 . 2 A Mathematical Framework for Feature Inference FINDER is a 3-step framework: 1) Dataset acquisition, 2) Feature construction, 3) Classification. We begin by assuming that the dataset \mathcal{D} , a subset of some nominal space U , is a random realization of a complete probability space (\Omega,\mathcal{F},\mathbb{P}) , without any assumptions on the underlying data distribution. U is usually a Euclidean space, but could be a manifold, spatio-temporal domain, etc. Formally, we view this realization of the random field through a map v_{1}:\Omega\to U . We then use another map v_{2} to map \mathcal{D} to some apt Hilbert space \mathcal{H} , such that different classes get mapped to disjoint regions in \mathcal{H} . Readers may recognize this as a standard feature construction task or kernel trick. The composition v\coloneqq v_{2}\circ v_{1} is called a stochastic feature (and is Bochner measurable A.1 ), if v\in L^{2}(\Omega,\mathcal{H}) . Classifiers are then simply maps from \mathcal{H} to \{0,1\} : usually via a machine from \mathcal{H} to [0,1] , with a separatrix t\in(0,1) . Figure 1 summarizes FINDER as a whole, but good features make Step 3 trivial, so our focus is on the composed Steps 1 and 2: the creation and computability of stochastic features. The novelty of our work is in that our features directly incorporate the stochasticity through which \mathcal{D} is generated. Thus, most binary classification problems fall within the ambit of our framework, while it becomes especially well-suited to handling noisy datasets. FINDER is agnostic to the choice of (\Omega,\mathcal{F},\mathbb{P}) if it is a complete probability space. \mathcal{H}=L^{2}([a,b]) or \mathbb{R}^{F} are usual choices if F is large. {\lx@inpgf@ignorespaces\Omega} {\lx@inpgf@ignorespaces\{0,1\}} {\lx@inpgf@ignorespaces U} {\lx@inpgf@ignorespaces\mathcal{H}} \scriptstyle{\lx@inpgf@ignorespaces 1)} \scriptstyle{\lx@inpgf@ignorespaces v_{1}} FINDER \scriptstyle{\lx@inpgf@ignorespaces v} \scriptstyle{\lx@inpgf@ignorespaces 2)\text{ }v_{2}} \scriptstyle{\lx@inpgf@ignorespaces 3)} Random Field Raw data Featured data { 0 , 1 } Class Figure 1: A schematic for FINDER and a visual perspective on classification as a multi-stage process. Unfortunately, constructing stochastic features v can be prohibitively expensive. However, our goal is classification and the eigen-decomposition of v via the KLE provides an

Read the complete text