Interpretable Feature Interaction via Statistical Self-supervised Learning on Tabular Data
Xiaochen Zhang, Haoyi Xiong · 2025
Read Interpretable Feature Interaction via Statistical Self-supervised Learning on Tabular Data by Xiaochen Zhang, Haoyi Xiong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18048.txt.
Introduction
Table data in fields like medicine demand models that are both accurate and easy to explain. Xiaochen Zhang and Haoyi Xiong present Spofe in 2025, a self taught pipeline that pulls features from tables while keeping its reasoning open to view. It first builds teaching signals with a kernel method that can catch curved patterns plain averages miss.
Those signals are then distilled into short formulas built from multiplied terms, which people can read. A knockoff based screen with strict error control picks which features truly matter. Across many real data sets, Spofe beat rival feature pickers at both sorting and forecasting tasks. The authors say the mix of power and clarity suits high stakes work.
Catalog record
| CREATOR | Xiaochen Zhang, Haoyi Xiong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18048
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interpretable Feature Interaction via Statistical Self-supervised Learning on Tabular Data Xiaochen Zhang, Haoyi Xiong Abstract In high-dimensional and high-stakes contexts, ensuring both rigorous statistical guarantees and interpretability in feature extraction from complex tabular data remains a formidable challenge. Traditional methods such as Principal Component Analysis (PCA) reduce dimensionality and identify key features that explain the most variance, but are constrained by their reliance on linear assumptions. In contrast, neural networks offer assumption-free feature extraction through self-supervised learning techniques such as autoencoders, though their interpretability remains a challenge in fields requiring transparency. To address this gap, this paper introduces Spofe , a novel self-supervised machine learning pipeline that marries the power of kernel principal components for capturing nonlinear dependencies with a sparse and principled polynomial representation to achieve clear interpretability with statistical rigor. Underpinning our approach is a robust theoretical framework that delivers precise error bounds and rigorous false discovery rate (FDR) control via a multi-objective knockoff selection procedure; it effectively bridges the gap between data-driven complexity and statistical reliability via three stages: (1) generating self-supervised signals using kernel principal components to model complex patterns, (2) distilling these signals into sparse polynomial functions for improved interpretability, and (3) applying a multi-objective knockoff selection procedure with significance testing to rigorously identify important features. Extensive experiments on diverse real-world datasets demonstrate the effectiveness of Spofe , consistently surpassing KPCA, SKPCA, and other methods in feature selection for regression and classification tasks. Visualization and case studies highlight its ability to uncover key insights, enhancing interpretability and practical utility. † † equal-contributors: These authors contributed equally to this work. This work was conducted through an independent collaboration between the two authors and bears no connection to the second author’s affiliation. 1 Introduction In the burgeoning field of data science, unsupervised feature extraction and selection have become foundational to exploratory data analysis and advanced analytics Hyvärinen and Oja (2000) ; Tipping and Bishop (1999) ; Van der Maaten and Hinton (2008) . These techniques not only reduce dimensionality and simplify complex data into more interpretable formats but also significantly enhance the performance of predictive models across various domains. For example, in healthcare, unsupervised feature extraction improves diagnostic accuracy by identifying key biomarkers from extensive medical datasets. In finance, it helps detect anomalies such as fraudulent transactions by isolating essential features from large-scale data. In retail, it supports customer segmentation and personalization by analyzing purchase histories, allowing businesses to tailor their services. These examples highlight the transformative potential of feature extraction in uncovering actionable insights from complex datasets across industries. Traditional methods like kernel Principal Component Analysis (KPCA) Schölkopf et al. (1998) was introduced, applying a nonlinear mapping to a higher-dimensional space before performing PCA Tipping and Bishop (1999) . This allows it to capture complex relationships that traditional PCA overlooks, making it well-suited for analyzing nonlinear behaviors and feature interactions in data. However, challenges remain, such as the need for precise kernel selection, vulnerability to overfitting, and significant computational demands that hinder scalability. Additionally, the transformation in Kernel PCA makes component interpretation more difficult, as the underlying features become less transparent. These challenges emphasize the need for more adaptable, assumption-free unsupervised learning methods that can better handle modern data complexities. One critical aspect of modern data analysis that demands such interpretability is the modeling of feature interactions. Feature interactions are a critical aspect of understanding complex datasets, as they capture how multiple features jointly influence outcomes in nonlinear or non-additive ways. Research on feature interactions has proposed diverse approaches, including statistical methods, information-theoretic measures, and machine learning techniques. Statistical approaches, such as functional ANOVA decomposition Hooker (2004) and rule-based ensemble models Friedman and Popescu (2008) , focus on quantifying interaction strength but struggle to handle high-dimensional, nonlinear, and high-order interactions. Information-theoretic methods, like mutual information-based feature selection Zeng et al. (2015) ; Liang et al. (2019) , detect interaction relevance but often lack explicit mathematical representations and rely heavily on supervised learning. Machine learning techniques, such as neural networks and multifactor dimensionality reduction (MDR) McKinney et al. (2006) ; de González and Cox (2007) , excel at identifying nonlinear interactions but are often computationally expensive and suffer from a lack of interpretability due to their ”black-box” nature. Table 1 : Comparison of Feature Learning Methods. The Fisher’s ratio refers to the ratio of between-class variance and within-class variance. Method Feature Interactions Interpretation (Un)supervision PCA N/A N/A Maximize variance Sparse PCA N/A Feature Attribution Maximize variance LDA N/A N/A Maximize the Fisher’s ratio Sparse LDA N/A Feature Attribution Maximize the Fisher’s ratio KPCA Kernel function N/A Maximize variance Sparse KPCA Kernel function Sample Influence Maximize variance KDA Kernel function N/A Maximize the Fisher’s ratio Sparse KDA Kernel function Sample Influence Maximize the Fisher’s ratio SDA with Poly. features Polynomial features Feature Attribution Maximize the Fisher’s ratio Our Work ( Spofe ) Polynomial features Feature Attribution Maximize variance To address these limitations, we introduce Spofe , a novel self-supervised machine learning pipeline designed for interpretable and statistically significant feature interaction detection. Spofe leverages the ability of KPCA to capture complex, nonlinear structures within data while addressing its limitations in interpretability by integrating a Knockoff filter framework. By combining these approaches, Spofe enables the detection of significant feature interactions in tabular data with both computational efficiency and statistical rigor, specifically tailored for scenarios where label information is unavailable. Spofe achieves this through three key components: self-supervised signal generation , multi-objective Knockoff selection procedure , and significance testing . Experiments on real-world datasets validated the effectiveness of this pipeline. Our contributions are as follows: • We propose to approximate kernel principal components (KPCs) using sparse polynomial functions, enabling self-supervised detection of significant feature interactions. This approach reframes the challenge of interpretability as a selective inference problem within a sparse polynomial space. By doing so, it bridges the gap between the complexity of kernel-based methods and the need for interpretable feature interaction modeling, allowing for rigorous hypothesis testing on polynomial coefficients. This enables the direct identification of both individual features and their significant interactions that contribute to the underlying data structure. Table 1 presents a comparison of various feature learning methods, highlighting their feature interactions, interpretations, and the ability to capture nonlinearity in kernel space, with a particular focus on how our work enhances these aspects through feature attribution together with polynomial expansions in unsupervised settings. • Spofe integrates statistical self-supervised signal generation via KPCA, Knockoff methods for multi-objective feature selection, and hypothesis testing into a unified framework tailored for feature interaction analysis. First, it projects the data onto the top kernel principal components to capture complex interaction patterns in an unsupervised manner. Next, it constructs synthetic knockoff statistics, weighting them by the proportion of variance explained by each principal component, to identify not only individual significant features but also their interactions across multiple dimensions. Finally, it estimates p -values for polynomial features within a hypothesis testing framework, achieving statistically valid and interpretable detection of feature interactions. To our knowledge, this is the first approach to combine self-supervised signals and Knockoff methods within a sparse polynomial space to detect and interpret significant feature interactions in tabular data. • We validate Spofe through extensive experiments on diverse real-world datasets, highlighting its versatility