CGPT: Cluster-Guided Partial Tables with LLM-Generated Supervision for Table Retrieval CCS: Information systems Content analysis and feature selection

Tsung-Hsiang Chou, Chen-Jui Yu, Shui-Hsiang Hsu, Yao-Chung Fan · 2026

Read CGPT: Cluster-Guided Partial Tables with LLM-Generated Supervision for Table Retrieval CCS: Information systems Content analysis and feature selection by Tsung-Hsiang Chou, Chen-Jui Yu, Shui-Hsiang Hsu, Yao-Chung Fan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15849.txt.

Introduction

Tsung-Hsiang Chou, Chen-Jui Yu, Shui-Hsiang Hsu, and Yao-Chung Fan improve table search in a 2026 arXiv paper. General embedding models do well at text retrieval. Tables are harder for them. Structured content leads to semantic compression and query-table mismatch. Recent methods generate synthetic queries with language models to help. They often pick partial tables by heuristic, though. They also rarely use the synthetic queries as training supervision.

Their framework is CGPT, built on cluster-guided partial tables. The partial views are chosen by clustering, not guesswork. LLM-generated supervision then improves the embedding model itself. Table retrieval matters wherever data live in spreadsheets and reports. Better matching between questions and tables saves analysts real time.

Catalog record

CREATORTsung-Hsiang Chou, Chen-Jui Yu, Shui-Hsiang Hsu, Yao-Chung Fan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-15849

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CGPT: Cluster-Guided Partial Tables with LLM-Generated Supervision for Table Retrieval CCS: Information systems Content analysis and feature selection Tsung-Hsiang Chou, Chen-Jui Yu, Shui-Hsiang Hsu, Yao-Chung Fan Abstract . General-purpose embedding models have demonstrated strong performance in text retrieval but remain suboptimal for table retrieval, where highly structured content leads to semantic compression and query–table mismatch. Recent LLM-based retrieval augmentation methods mitigate this issue by generating synthetic queries, yet they often rely on heuristic partial-table selection and seldom leverage these synthetic queries as supervision to improve the embedding model. We introduce CGPT , a training framework that enhances table retrieval through LLM-generated supervision . CGPT constructs semantically diverse partial tables by clustering table instances using K-means and sampling across clusters to broaden semantic coverage. An LLM then generates synthetic queries for these partial tables, which are used in hard-negative contrastive fine-tuning to refine the embedding model. Experiments across four public benchmarks (MimoTable, OTTQA, FetaQA, and E2E-WTQ) show that CGPT consistently outperforms retrieval baselines, including QGpT, with an average R@1 improvement of 16.54%. In a unified multi-domain corpus setting, CGPT further demonstrates strong cross-domain generalization and remains effective even when using smaller LLMs for synthetic query generation. These results indicate that semantically guided partial-table construction, combined with contrastive training from LLM-generated supervision, provides an effective and scalable paradigm for large-scale table retrieval. Our code is available at https://github.com/yumeow0122/CGPT . 1. Introduction Tables are a central medium for storing and communicating structured information across domains such as finance, science, logistics, and public knowledge bases. With the rapid growth of large web-scale table corpora, effective table retrieval has become increasingly important for downstream tasks, e.g., table question answering. However, general-purpose embedding models trained on textual corpora exhibit strong generalization in text retrieval, yet face substantial challenges when extended to table retrieval. Encoding an entire table into a single vector thus induces semantic compression, resulting in degraded retrieval performance. Enhancing representation learning ( 1 ; 7 ; 5 ) for table retrieval is therefore a central objective. Beyond task-specific table embedding fine-tuning ( 1 ; 7 ) , recent work has explored the use of large language models (LLMs) to provide additional supervision for table retrieval without modifying the embedding model directly. A representative approach in this direction is QGpT ( 5 ) , which selects the first k rows of a table to form a partial table and prompts an LLM to generate synthetic queries (SQ). These synthetic queries serve as enhanced table representations and can improve retrieval accuracy. While effective, QGpT has two key limitations: (1) selecting only the first rows assumes they are representative, which does not hold when relevant information appears in later rows; and (2) the generated synthetic queries are seldom used as training supervision to improve the embedding model itself, leaving potential performance gains unrealized. Figure 1. Overall workflow of CGPT . The framework consists of four main stages: (1) clustering-based partial table generation; (2) synthetic query generation; (3) hard negative sampling; and (4) model fine-tuning. To address these challenges, we propose CGPT , a framework that enhances table retrieval through LLM-generated supervision . CGPT applies K-means clustering to group table rows into semantically coherent subsets and samples from each cluster to construct partial tables with broader coverage of table attributes and instances. Synthetic queries generated by an LLM are then leveraged in hard-negative contrastive fine-tuning, enabling the embedding model to better retrieve tables even when relevant information is sparsely located across rows. We evaluate CGPT on four public benchmarks—MimoTable, OTTQA, FetaQA, and E2E-WTQ—and observe consistent improvements over retrieval augmentation baselines, with a mean R@1 gain of 16.54%. Additional experiments on a unified multi-domain corpus confirm cross-domain generalization, and experiments with smaller LLMs show that CGPT remains cost-efficient without sacrificing retrieval effectiveness. 2. Related Work 2.1. Table Retrieval Table retrieval has emerged as an important problem in natural language processing. TAPAS ( 2 ) extends the BERT architecture with table-aware positional encodings (e.g., row and column indices) and pre-trains on millions of Wikipedia tables, establishing a strong foundation for table QA. Dense Table Retrieval (DTR) ( 1 ) adopts dense vector retrieval by encoding queries and tables separately with two TAPAS encoders and computing similarity via inner products. DTR further incorporates the Inverse Cloze Task ( 4 ) and hard negative sampling to improve retrieval quality. A common limitation of these approaches is the use of a single vector to represent an entire table. For large tables, this design can cause excessive semantic compression. 2.2. Representation Augmentation Recent work explores synthetic query generation to strengthen table representations. QGpT ( 5 ) adopts a partial-table strategy by selecting the first 10 rows and generating synthetic queries using a large language model; the partial table and generated queries jointly define the table representation. This reduces input length and yields more focused supervision, improving zero-shot performance of general-purpose embedding models. GenSearch ( 3 ) instead generates pseudo tables conditioned on queries, further illustrating the promise of generative retrieval. Building on these ideas, we aim to construct partial tables that more comprehensively cover the semantic space of the original table. Our method, CGPT , clusters table instances via K-means and samples across clusters to form semantically diverse partial tables with improved coverage. 3. Method As illustrated in Figure 1 , CGPT follows a four-stage training pipeline: clustering–based partial table generation, synthetic query generation, hard negative sampling, and contrastive fine-tuning. Given a table T=\{H,I_{1},\dots,I_{m}\} with header H and m instances, we construct training data and optimize the embedding model through the steps below. 3.1. Clustering-based Partial Table Generation To capture a broader portion of the table’s semantic space, we generate multiple representative partial tables using K-means, called KPT (K-means Partial Table). This process includes instance embedding, clustering, and partial-table formation. Instance Embedding For each instance I_{i} in table T , we encode it using a pretrained embedding model, producing a set of instance embedding vectors \mathcal{e}=\{\mathbf{e}_{1},\mathbf{e}_{2},...,\mathbf{e}_{m}\} . K-means Clustering To ensure that the constructed partial tables capture the table’s full semantic space, we apply K-means clustering over \mathcal{e} . The number of clusters k is determined adaptively based on the table size: (1) k=\min\left(\left\lceil\frac{m}{r}\right\rceil,k_{\max}\right) Forming K-means Partial Table From each cluster C_{j} , we randomly sample s instances to construct the j -th partial table: (2) KPT_{j}=\{H\}\cup\{I_{i}\mid I_{i}\in\text{Sample}(C_{j},s)\} This procedure yields k partial tables per original table, with each KPT reflecting the semantics of a specific cluster. 3.2. Synthetic Query Generation For each partial table KPT_{j} , we prompt a LLM to generate n_{q} synthetic queries. The generation process is model-agnostic and only requires the ability to produce natural-language queries conditioned on the content of the partial table. For reproducibility, the exact prompt template is provided in Appendix A . 3.3. Hard Negative Sampling and Fine-tuning To strengthen the model’s ability to distinguish semantically similar but incorrect tables, we integrate hard negative sampling with contrastive fine-tuning. 3.3.1. Hard Negative Sampling For each synthetic query q , we compute cosine similarity between q and all KPTs from other tables using a pretrained embedding model, selecting the top- h most similar but incorrect ones as hard negatives to form \mathcal{HN} . These negatives, which are easily confused with the true positive, provide strong supervision for improving

Read the complete text