TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering

Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang · 2026

Read TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering by Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21656.txt.

Introduction

Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, and Qiong Zhang cluster tabular data with a network fitted on synthetic tasks. Clustering tables is hard because columns mix types, the hidden generating process varies, and tricks rarely transfer between datasets. Prior-fitted networks have done well at supervised table tasks by mimicking Bayesian inference.

The authors extend that idea to clustering, where no labels exist at all. A network trained on a broad synthetic prior can cluster a new table in one pass. Transferable clustering would help analysts facing fresh tables daily, from hospital records to sales data, with no time to tune per dataset.

Catalog record

CREATORTianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21656

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang Abstract Clustering tabular data is a fundamental yet challenging problem due to heterogeneous feature types, diverse data-generating mechanisms, and the absence of transferable inductive biases across datasets. Prior-fitted networks (PFNs) have recently demonstrated strong generalization in supervised tabular learning by amortizing Bayesian inference under a broad synthetic prior. Extending this paradigm to clustering is nontrivial: clustering is unsupervised, admits a combinatorial and permutation-invariant output space, and requires inferring the number of clusters. We introduce TabClustPFN , a prior-fitted network for tabular data clustering that performs amortized Bayesian inference over both cluster assignments and cluster cardinality. Pretrained on synthetic datasets drawn from a flexible clustering prior, TabClustPFN clusters unseen datasets in a single forward pass, without dataset-specific retraining or hyperparameter tuning. The model naturally handles heterogeneous numerical and categorical features and adapts to a wide range of clustering structures. Experiments on synthetic data and curated real-world tabular benchmarks show that TabClustPFN outperforms classical, deep, and amortized clustering baselines, while exhibiting strong robustness in out-of-the-box exploratory settings. Code is available on Github . K eywords End-to-end \cdot Foundation model \cdot In-context learning \cdot Unsupervised 1 Introduction Clustering is a fundamental technique often applied to tabular datasets, which are ubiquitous across scientific research and industry. By grouping similar data points, it reveals important latent structure that can be used for downstream modeling. Ideally, clustering serves as an agile, out-of-the-box exploratory tool: a practitioner receives a dataset, lacks prior knowledge of the cluster count, has limited opportunity for hyperparameter tuning, and expects immediate insight. This setting arises in applications from genetic data analysis ( Eisen et al., 1998 ) to customer segmentation ( Kansal et al., 2018 ) . Despite its ubiquity, reliably clustering tabular data under these conditions remains a persistent challenge due to the inherent trade-offs in existing methods. The first class comprises classical probabilistic and distance-based approaches. While computationally efficient, these methods operate directly on the input features and rely on rigid geometric assumptions to separate data: k -means ( Lloyd, 1982 ) and Gaussian mixture models (GMMs) assume convex or Gaussian shapes, DBSCAN ( Ester et al., 1996 ) relies on uniform density, and hierarchical clustering ( Johnson, 1967 ) is highly sensitive to the choice of metric or linkage criterion. These rigid biases often fail to capture the heterogeneous structures of real-world data. To address these geometric limitations, a second class of methods, manifold and representation learning approaches, maps data into latent spaces where cluster structure is more separable. Spectral clustering ( Ng et al., 2001 ) captures non-convex manifolds via Laplacian eigenspace embeddings, while recent deep clustering methods ( Rauf et al., 2025 ; Ren et al., 2024 ) learn complex nonlinear representations with neural networks. However, both frame clustering as a dataset-specific optimization problem, leading to high computational cost: spectral clustering relies on eigen decomposition with cubic complexity in the number of samples, and deep methods require computationally expensive model training for each dataset. Thus, they are too slow for exploratory use and lack a transferable inductive bias. A third paradigm consists of amortized clustering methods ( Lee et al., 2019a ; Lee et al., 2019b ; Pakman et al., 2020 ) , which train neural networks to predict cluster assignments iteratively. These approaches achieve zero-shot inference, but typically approximate simple priors such as GMMs. As such, they inherit the limitations of the parametric models they emulate. Consequently, a gap remains for a method that is both fast at inference without per-dataset optimization and sufficiently expressive to capture complex, non-standard cluster geometries beyond simple parametric priors. Furthermore, all categories of existing methods typically require users to manually specify parameters—such as the number of clusters—without ground truth to validate these decisions. Towards addressing these limitations, we draw inspiration from the emerging class of Prior-Fitted Networks (PFNs) ( Müller et al., 2022 ) in supervised learning. PFNs, such as TabPFN ( Hollmann et al., 2025 ) , have demonstrated the capability to break the speed-expressivity tradeoff as well as to adapt automatically to the data structure without any hyperparameter tuning. Extending the PFN paradigm to clustering, however, is nontrivial. Unlike supervised learning, clustering targets are set-valued and permutation-invariant, requiring the model to handle the label-switching problem. Furthermore, the absence of ground-truth labels makes it challenging to define a training objective that jointly infers cluster assignments and cardinality for the entire dataset. We solve these challenges and introduce TabClustPFN , a PFN for tabular clustering, in this work. Our approach explicitly decomposes the problem into two components: a Cardinality Inference Network that predicts the number of clusters from the data, and a Partition Inference Network that predicts assignments conditioned on that estimate. Both operate through in-context learning (ICL) and require no per-dataset optimization. TabClustPFN therefore addresses the core limitations of existing clustering methods: 1. Automated model selection: It performs end-to-end clustering while automatically inferring the number of clusters. This solves the “no ground truth” validation problem by leveraging a learned prior rather than manual heuristics, while still supporting user-provided constraints when available. 2. Interactive speed via amortization: Unlike methods that learn geometry per-dataset, TabClustPFN approximates the posterior in a single forward pass. As a result, on datasets with up to 1,000 points, TabClustPFN is up to 500\times faster than Spectral clustering even when the number of clusters is unknown. 3. Flexible inductive bias: It naturally handles heterogeneous tabular data (numerical and categorical) without requiring handcrafted distance metrics or preprocessing, making it highly robust in zero-shot settings. Moreover, TabClustPFN is highly interpretable, allowing users to extract useful representations and cluster prototypes. Beyond simple assignments, it provides qualitative and visual insights into cluster structure, such as measure of centrality and hierarchical relationships. Figure 1: Comparison of clustering time and median ARI rank (lower is better) . TabClustPFN achieves the best median ARI rank while maintaining computational time comparable to simple methods, and faster than other PFN-based approaches. Finally, to evaluate TabClustPFN , we introduce a curated benchmark for tabular clustering comprising 44 real datasets with up to 1{,}000 samples, 64 features, and 10 clusters, spanning both numerical and categorical variables. We compare TabClustPFN against classical and deep learning baselines on this real data benchmark as well as simulated data benchmarks, demonstrating state-of-the-art performance across datasets (Fig. 1 ), along with ablation studies validating our design choices. 2 An overview of TabPFN TabPFN ( Hollmann et al., 2025 ) is a canonical Prior-Fitted Network (PFN) that servers as a tabular foundation model for supervised learning that casts prediction as Bayesian inference. Given a dataset {\mathcal{D}}=\{(X_{i},Y_{i})\}_{i=1}^{n} drawn IID from an unknown distribution \pi^{*} and a prior p over a space of data-generating distributions \Pi , TabPFN approximates the posterior predictive distribution (PPD): p(y|x,{\mathcal{D}})=\int_{\Pi}\pi(y|x)\,p(\pi|{\mathcal{D}})\,d\pi. To achieve this without intractable integrals, TabPFN uses PFN: a transformer is pretrained to approximate this PPD directly, enabling ICL. At test time, the model parameters are fixed; it processes the dataset {\mathcal{D}} as a sequence of tokens in the context window and predicts the target y for a query x in a single forward pass. Another core innovation of TabPFN lies in its training data. Instead of using real-world datasets, the model is

Read the complete text