Towards Evaluating Data Priors for Tabular Foundation Models
Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter · 2026
Read Towards Evaluating Data Priors for Tabular Foundation Models by Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29241.txt.
Introduction
Turkmen, Kaya, Pfefferle and Hutter ask how much the data-generating priors behind tabular foundation models shape what those models learn. Such models are pretrained on synthetic tasks drawn from a prior, a recipe for inventing tables, so the prior defines the world the model practises on. Priors are rarely evaluated on their own, making their effect hard to isolate.
The team built a single interface to publicly available priors from recent tabular models, plus priors built from real datasets. They generated training tasks from each prior, trained the same architecture under a fixed protocol, and evaluated on shared downstream classification tasks. Different priors, they find, favour different downstream behaviours: some give stronger absolute performance, others more consistent relative rankings across datasets. Data-level similarity between priors only partially explained downstream behaviour. The study offers a method for comparing priors on equal footing, with code released publicly.
Catalog record
| CREATOR | Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.29241
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Evaluating Data Priors for Tabular Foundation Models Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter Abstract Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated-task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data-level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM-Playground/tree/prior-dev . † † affiliationnotice: Equal contribution 1 Introduction Since the introduction of TabPFNv1 ( 5 ) , tabular foundation models have emerged as a promising direction for in-context learning on structured data. These models are pretrained on large collections of tabular tasks, where the data-generating prior defines the task distribution seen during training. As a result, prior design is a central component of tabular foundation models, shaping the downstream predictive behavior of the resulting model. Despite this central role, priors are rarely evaluated as independent components. Models such as TabPFN, TabForestPFN ( 2 ) , and TabICL ( 11 ) are introduced together with specific choices of prior generation, architecture, optimization, and training protocol. This makes it challenging to isolate the effect of the prior itself. To better understand their role, we take a first step towards exploring how different data-generating priors affect downstream behavior under a controlled experimental setting. Using this framework, we study whether heterogeneous priors can be compared under a unified setting, whether measurable properties of generated tasks relate to downstream behavior, and how much prior choice matters when architecture and optimization are fixed. Our contributions are as follows: (i) we implement a unified interface for task generation across priors from TabPFNv1, TabICL, TICL ( 8 ) , TabForestPFN, and our own real-data tasks; (ii) we evaluate priors as independent components under a fixed tabular foundation model architecture and training protocol, comparing them through data-level similarity and downstream predictive performance; and (iii) we show that different priors favor different downstream behaviors across datasets, while data-level statistical similarity only partially explains downstream behavior. 2 Related Work Recent tabular foundation models such as TabPFNv1, TabICL, TICL, and TabForestPFN each introduce their own prior-generation mechanisms, described in their respective works. These systems have expanded the range of priors used in practice, but prior design is often treated as a component of the overall model rather than studied in isolation. MITRA ( 12 ) , on the other hand, provides a more direct study of prior design and task generation. The study proposes a way to identify performance, diversity and distinctiveness of priors and combines priors with these characteristics for model training. MITRA characterizes priors to combine them; we isolate them under a fixed architecture to compare them, and include real-data priors that MITRA does not. Furthermore, other work ( 6 ) shows that generalization can emerge even from a single real table when diverse tasks are constructed by varying targets and feature subsets. 3 Towards a Unified Evaluation Pipeline for Data Priors To study priors as independent components, we introduce a unified evaluation pipeline that standardizes data generation, model training, and downstream evaluation across methods. We build this work as an extension to the TFM-Playground ( 9 ) . Each prior generates a fixed budget of training tasks used to train an identical nanoTabPFN ( 10 ) model. Priors are then evaluated through both data-level statistics and downstream predictive performance on datasets drawn from TabArena ( 3 ) as shown in Appendix Table 3 . Prior Families Generate 250k Tasks Data Characterization Train Fixed nanoTabPFN Downstream Evaluation and Similarity Comparison Figure 1 : Pipeline for studying data priors through generated-data characterization and downstream evaluation. 3.1 Unified Prior Generation We unify priors from multiple libraries under a single interface. TabICL: generates tasks by sampling latent Gaussian variables and propagating them through layered Structural Causal Model (SCM) inspired transformations that introduce dependencies between variables. Across different TabICL variants, layers are parameterized by MLP mappings, tree-based mappings, or a mixture of both. Features and targets are then selected from intermediate variables of the resulting representation space. TabForestPFN: generates tasks by fitting standard machine learning models to randomly sampled data and using their predictions on unseen inputs as targets. Depending on the selected variant, generators are based on decision trees, nearest neighbors, or random projection classifiers, yielding tasks with model-specific decision boundaries. TabPFNv1: uses a prior implementation from a public GitHub repository based on TabPFNv1 1 1 1 TabPFNv1 prior repository . It employs function based priors such as MLPs, Gaussian processes, and mixtures thereof to sample target functions evaluated on random inputs. TICL: uses a closely related function formulation to the TabPFNv1 prior, based on MLP and Gaussian process generators, with alternative preprocessing and target conversion routines. Real Data: uses a controlled real-data task-generation pipeline inspired by recent real-data approaches such as RealTabPFN ( 4 ) and TabDPT ( 7 ) . The pipeline consists of three stages: (i) Dataset curation: datasets are collected from OpenML, cleaned, preprocessed, and stored together with metadata. (ii) Leakage reduction: duplicate and near duplicate datasets are removed by checking against the evaluation pool and within the training pool using statistical fingerprinting based on hashes, dataset dimensions, target statistics, and feature distribution summaries. (iii) Task construction: remaining datasets are partitioned into classification, regression, and mixed target pools. Training tasks are generated by sampling datasets, selecting either the original target or a randomly chosen task-compatible column, splitting rows into context and query sets. Task-level preprocessing is then fitted on the context rows only before being applied to both sets. 3.2 Experimental Generation Setup For all priors, we use the default hyperparameters of their publicly available reference implementations. Our goal is not to optimize each prior individually, but rather to compare different priors under a consistent and controlled setup. Future work could examine whether prior-specific hyperparameter tuning changes the relative performance of different priors. We use a fixed data generation configuration to ensure comparability. Each prior generates 250,000 tasks with up to 512 samples and up to 16 features. For most priors, the feature count is fixed. For the Real Data prior, the number of features and sequence length are capped by dataset availability. For TabICL priors, the minimum feature constraint can behave as a soft constraint, since the original implementation may remove near-constant features during post-generation filtering. These limits were chosen to keep experiments computationally feasible, particularly as larger sequence lengths and higher-dimensional feature spaces substantially increase memory and runtime requirements. Experiments were conducted with an NVIDIA GeForce RTX 2080 Ti GPU. 3.3