Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets
M. Ross Kunz, John Merickel, Keith Wilson · 2026
Read Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets by M. Ross Kunz, John Merickel, Keith Wilson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30289.txt.
Introduction
M. Ross Kunz, John Merickel and Keith Wilson describe numeric tables through survey descriptors. Descriptors enter a shared vector space with a sentence transformer, and likeness is measured via Canonical Correlation Analysis. A penalized CCA method recovers sparse variable matches, showing which descriptors drive matching without shared names or conventions. Differential privacy is applied before embedding, allowing use in private contexts without access to raw records. Across 15 sets spanning general, materials and graphite data, it reaches a P@1 score of nine tenths, with retrieval and cluster structure holding across ablations.
Catalog record
| CREATOR | M. Ross Kunz, John Merickel, Keith Wilson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73606 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30289
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets M. Ross Kunz, John Merickel, Keith Wilson Abstract Numeric tabular datasets are the dominant data format in scientific practice, yet large language models lack native mechanisms for representing numeric datasets in a meaningful way across heterogeneous feature spaces. Existing approaches either target predictive modeling over individual datasets, which requires a shared set of variable definitions, or lack mechanisms for interpretable cross-dataset alignment. The proposed methodology characterizes numeric tabular datasets through structured exploratory data analysis descriptors, embeds those descriptors into a shared vector space using a pretrained sentence transformer, and quantifies cross-dataset similarity via Canonical Correlation Analysis (CCA). Furthermore, a penalized formulation of CCA is applied to recover sparse, interpretable variable-level correspondences between datasets, identifying which statistical descriptors or variable-level quantities drive cross-dataset alignment without requiring shared variable names or feature conventions. Differential privacy is optionally applied to the descriptor set prior to embedding, supporting deployment in sensitive data contexts without requiring access to raw observations at time of comparison. The methodology is evaluated across 15 datasets spanning general-purpose benchmarks, materials informatics, and nuclear-grade graphite characterization. Results demonstrate a total P @ 1 P@1 score of 0.9, with known nearest-neighbor retrieval and cluster structure remaining robust across embedding ablations and differential privacy budgets. The proposed framework provides a principled pathway for integrating heterogeneous numeric data into retrieval-augmented generation pipelines while preserving statistical context, with direct applications to data-driven algorithm selection and simulation model initialization for unknown datasets. keywords tabular data embedding, dataset similarity, canonical correlation analysis, retrieval-augmented generation 1 Introduction Numeric tabular datasets are the dominant data format in scientific practice, yet organizing, searching, and comparing such datasets across heterogeneous feature spaces remains a largely unsolved problem in data management and knowledge discovery. Scientific data repositories accumulate datasets that vary in sample size, variable composition, measurement scale, and domain context, making it difficult to determine which datasets are structurally related, which share distributional properties, or which are likely to respond similarly to a given modeling approach. Existing dataset search and profiling tools produce human-readable summaries or rely on schema matching over shared variable names, neither of which supports similarity comparison across datasets with disjoint feature spaces. This limitation is especially pronounced in natural science domains, where a single physical system may be characterized through multiple experimental modalities with incompatible variable definitions, and where the cost of redundant or mismatched modeling efforts is high. A principled representation of dataset-level statistical structure comparable across heterogeneous feature spaces, and interpretable in terms of the underlying data properties, would substantially advance the ability to organize, retrieve, and reason over large collections of scientific tabular data. Large language model (LLM) embedding spaces offer a potential pathway toward such a representation. By encoding text into vectors of common dimension, pretrained sentence transformers enable similarity comparisons across documents from disparate domains without requiring shared vocabulary or structure. This capability motivates a statistical embedding strategy: rather than representing a dataset by its raw observations or variable names, one characterizes it through structured exploratory data analysis (EDA) descriptors, serializes those descriptors into natural language sentences, and embeds the resulting sentences into the LLM’s semantic space. The central challenge is ensuring that this embedding process preserves the statistical structure of the underlying data. The proposed methodology computes individual statistical descriptors for each numeric column alongside multivariate summary quantities, which together serve as the basis for embedding tabular data into a shared vector space. These embeddings support hierarchical clustering and Canonical Correlation Analysis (CCA) across datasets, analogous to multi-view document embeddings ( 11 ) , even when data sources differ in dimension or variable composition. A penalized formulation of CCA is further employed to identify which variables or multivariate quantities drive cross-dataset correlation, yielding directly interpretable alignment structure. Differential privacy is optionally applied to the statistical descriptors prior to embedding, providing a principled mechanism for obfuscating sensitive summary information while preserving sufficient structure for meaningful cross-dataset comparison. Together, these components provide a means of interpretable tabular RAG that organizes heterogeneous numeric data while preserving statistical context. The contributions of this work are as follows: • A structured EDA pipeline that produces statistically grounded, sentence-serialized descriptors for numeric tabular datasets, with optional differential privacy for sensitive data contexts, formalized as a mapping from a raw data matrix to a collection of embedding vectors in a shared semantic space. • A cross-dataset similarity framework based on mean canonical correlation between EDA embedding collections, supporting nearest-neighbor retrieval and hierarchical clustering across datasets with heterogeneous and disjoint feature spaces. • A penalized CCA formulation yielding sparse canonical loadings that identify which statistical descriptors or variable-level quantities drive cross-dataset alignment, providing interpretability not available under standard CCA. • Empirical validation across 15 datasets spanning general-purpose benchmarks, materials informatics, and nuclear-grade graphite characterization, demonstrating a total P@1 score of 0.9 with nearest-neighbor retrieval. • A framework for retrieval-based algorithm and simulation model initialization, in which statistical similarity to cataloged datasets with known modeling outcomes supports data-driven selection of candidate methods for unknown datasets, connecting the proposed approach to meta-learning and AutoML pipelines. The remainder of this paper is organized as follows. Section 2 reviews related work on tabular representation learning, dataset search, and differential privacy for data publishing. Section 3 describes the EDA fingerprinting pipeline, embedding procedure, and penalized CCA similarity framework. Section 4 details the datasets used for evaluation. Section 5 presents nearest-neighbor retrieval results, hierarchical clustering analysis, and penalized CCA interpretability examples. Section 6 summarizes the findings and discusses directions for future work. 2 Related Work 2.1 Tabular Representation and Foundation Models Several lines of research have addressed the challenge of representing tabular data within learned model architectures. TabNet ( 3 ) introduced sequential attention over tabular features to enable instance-wise feature selection during training, improving interpretability for prediction tasks. SAINT ( 35 ) extended the transformer architecture to tabular data through inter-sample and inter-feature attention, demonstrating competitive performance on supervised learning benchmarks. TabPFNv2 ( 20 ) demonstrated that a transformer pretrained exclusively on synthetic tabular data achieves state-of-the-art predictive accuracy on small datasets via in-context learning. TabICL ( 32 ) extended this paradigm to large datasets through a two-stage column-then-row attention mechanism. 37 argue that such Large Tabular Models (LTMs) remain significantly underrepresented in the foundation model literature despite the dominance of tabular data across scientific domains. Each of these approaches targets predictive modeling performance on individual datasets and does not address the problem of representing or comparing datasets across heterogeneous feature spaces. 2.2 Dataset Search and Data Discovery The problem of organizing and retrieving datasets from large repositories has received sustained attention in the data management community. Dataset search over data lakes, collections of heterogeneous raw data assets stored without imposed schema, has been formalized as a problem of identifying semantically or structurally related tables from among thousands of candidates ( 29 ) . A central challenge in this setting is that related datasets often share neither variable names nor schema structure, requiring similarity measures that operate over content rather than metadata alone. Schema matching approaches address this by identifying correspondences between column names or value distributions across pairs of tables ( 33 ) , but these methods rely on