Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks [Experiment, Analysis & Benchmark]

Liane Vogel, Kavitha Srinivas, Niharika D’Souza, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz · 2026

Read Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks [Experiment, Analysis & Benchmark] by Liane Vogel, Kavitha Srinivas, Niharika D’Souza, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21696.txt.

Introduction

Vogel and five colleagues introduce TEmBed, a unified test bed for tabular foundation models, which aim to learn reusable representations of table data for retrieval, search, and prediction. Existing models are hard to compare because each is tested under its own task settings. TEmBed evaluates embeddings at cell, row, column, and table level, showing that the best model depends on the task and level, and offering practical guidance for choosing one.

Catalog record

CREATORLiane Vogel, Kavitha Srinivas, Niharika D’Souza, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97982
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21696

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks [Experiment, Analysis & Benchmark] Liane Vogel, Kavitha Srinivas, Niharika D’Souza, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz Abstract . Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as table retrieval, semantic search and table-based prediction. Despite the growing number of such models, it remains unclear which approach works best in practice, as existing methods are often evaluated under task-specific settings that make direct comparison difficult. To address this, we introduce TEmBed, the T abular E mbedding Test B ed , a comprehensive benchmark for systematically evaluating tabular embeddings across four representation levels: cell, row, column, and table. Evaluating a diverse set of tabular representation learning models, we show that which model to use depends on the task and representation level. Our results offer practical guidance for selecting tabular embeddings in real-world applications and lay the groundwork for developing more general-purpose tabular representation models. With Extended Technical Report † † authors: . PVLDB Reference Format: PVLDB, xx(x): XXX-XXX, xxxx. doi:XX.XX/XXX.XX † † This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing [email protected] . Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. xx, No. x ISSN 2150-8097. doi:XX.XX/XXX.XX PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/IBM/table-representation-evals . 1. Introduction Representation Learning on Relational Data. Tabular data is a core data modality in database systems and remains the dominant format for structured data management in practice. Consequently, representation learning for relational and tabular data has become an important research direction at the intersection of databases and machine learning. The goal is to embed tables in a way that captures semantic, structural, and statistical properties such that the resulting embeddings are reusable for downstream applications. As shown in Figure 1 , tabular embeddings must capture four different levels of structural information, each of which is useful for different tasks; e.g., duplicate detection requires row-level embeddings, while table discovery requires table-level embeddings. Figure 1 . Overview of TEmBed. We systematically evaluate tabular embeddings covering four representation levels: cell, row, column, and table on a diverse set of tasks across 69 datasets. Tabular Foundation Models. Recently, a growing number of table foundation models ( 37 ; 18 ; 6 ) have been introduced with the goal of learning universal representations of tabular data that transfer across tasks, domains, and datasets without requiring additional training. These models leverage large-scale pretraining on heterogeneous data sources, enabling them to produce cell-, row-, column-, or table-level embeddings for any new table. Similar developments in natural language processing have been accompanied by standardized evaluation frameworks, most notably the Massive Text Embedding Benchmark (MTEB) ( 36 ) , which enables systematic comparison of general-purpose text embedding models across diverse tasks and is the de facto standard for the field. However, no analogous benchmark exists for tabular embeddings . As a result, existing evaluations are narrow, task-specific, and use different embedding levels with incompatible protocols, making it difficult to comprehensively compare models ( 2 ) . Assessing the semantic and structural quality of learned embeddings is inherently challenging, as it often requires ground truth signals that vary across tasks and domains. For tabular models, this challenge is exacerbated by the different granularity levels of embeddings required for tables, which may require fundamentally distinct evaluation criteria: a good table-level embedding must capture global schema and content, while a good cell-level embedding must reflect local value semantics in context. Contributions. As our core contribution, we introduce TEmBed , a comprehensive benchmarking framework for tabular embeddings. Our framework evaluates models across cell-, row-, column-, and table-level tasks using a diverse collection of datasets, including some newly developed benchmarks. Further, TEmBed provides an open-source toolkit to easily extend both the supported models and benchmark datasets, enabling consistent measurements of model performance, time efficiency, and resource consumption across diverse tasks. By providing a common evaluation framework, TEmBed enables meaningful cross-task comparison, supports informed model selection, and establishes a foundation for developing more general-purpose tabular representation models. Our main contributions are: (1) We introduce the first comprehensive benchmarking framework for tabular embeddings that spans multiple datasets, tasks, and embedding levels (cell, row, column, table). We incorporate multiple existing public datasets and complete this selection by introducing multiple new datasets. (2) We provide an extensible open-source test bed with efficiency measurements that works across a large diversity of different embedding models. (3) We perform an comprehensive empirical study of state-of-the-art tabular embedding models, assessing their performance, computational cost, generalization across tasks, and offer practical guidance for model selection. 2. Background 2.1. Tabular Data Representations Table Encoders. Learning representations of tabular data has been an active research area over the past years, as surveyed by 2 . Early work applied the pre-training and fine-tuning paradigm to relational web-tables, using objectives such as masked entity recovery ( 11 ) , corrupt cell detection ( 21 ) or jointly training on tables and surrounding text ( 50 ) . Most of these models targeted specific downstream tasks, such as table question answering ( 17 ) or column type annotation ( 43 ) . More recent work such as HyTrel ( 6 ) takes a more universal perspective, modeling tables as hypergraphs to produce representations at all four granularity levels - cells, rows, columns and tables. A simpler but widely adopted approach to produce tabular embeddings foregoes tabular-specific pretraining altogether: table content is serialized into a flat text sequence, typically by concatenating column headers and cell values, and passed to a general-purpose text embedding model ( 22 ; 3 ) . LLMs for Generative Table Reasoning. A parallel line of work adapts large language models to tabular data for generative and conversational reasoning tasks. Models such as TableLlama ( 51 ) and Table-GPT ( 30 ) are fine-tuned to support natural language interaction with tables, enabling question answering, SQL generation, summarization, and interactive data processing. These models target tasks end-to-end generative use cases and operate in a prompt-driven setting, making them complementary to the task-agnostic embeddings evaluated in this benchmark. Tabular Prediction. A distinct line of work focuses on tabular prediction, the task of predicting a target column from the remaining columns of a table, analogous to classical supervised learning on structured data. Traditionally dominated by gradient-boosted tree methods such as XGBoost ( 7 ) , this field has seen growing interest in foundation model approaches. TabPFN ( 18 ) pioneered in-context learning for tabular prediction, training a Transformer on synthetic and real-world data to perform classification or regression at inference time without gradient updates. TabICL ( 37 ) extended this paradigm with improved scalability to larger datasets. Other approaches fine-tune large language models on broad collections of real-world tables ( 14 ) by serializing rows as text. Further approaches incorporate semantic signals — such as column headers and categorical value names — by encoding table content with pretrained text ( 41 ) or graph-based models or incorporating external knowledge

Read the complete text