CRAFT : Training-Free Cascaded Retrieval for Tabular QA
Jianxi Gao · 2025
Read CRAFT : Training-Free Cascaded Retrieval for Tabular QA by Jianxi Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14984.txt.
Introduction
Adarsh Singh and co-authors present, in a 2025 paper, a training-free way to find the right tables to answer a question. Open table question answering must pull relevant tables from a large pool. Heavy search models do this well but cost a lot to run and must be retrained for each new field.
Their system, CRAFT, works in stages. A fast, sparse search first narrows the pool, and only then do heavier models re-rank the short list. To sharpen matches, they enrich each table with a title and summary written by a language model. The system beat rival searchers on one benchmark and held up well on a harder one that needs reasoning across tables and text, all without training.
Catalog record
| CREATOR | Jianxi Gao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14984
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CRAFT : Training-Free Cascaded Retrieval for Tabular QA Jianxi Gao Abstract Table Question Answering (TQA) involves retrieving relevant tables from a large corpus to answer natural language queries. Traditional dense retrieval models, such as DTR and ColBERT, not only incur high computational costs for large-scale retrieval tasks but also require retraining or fine-tuning on new datasets, limiting their adaptability to evolving domains and knowledge. In this work, we propose CRAFT , a cascaded retrieval approach that first uses a sparse retrieval model to filter a subset of candidate tables before applying more computationally expensive dense models and neural re-rankers. Our approach achieves better retrieval performance than state-of-the-art (SOTA) sparse, dense, and hybrid retrievers. We further enhance table representations by generating table descriptions and titles using Gemini Flash 1.5 Team et al. 2025 . End-to-end TQA results using various Large Language Models (LLMs) on NQ-Tables Herzig et al. 2021 , a subset of the Natural Questions Dataset, demonstrate CRAFT effectiveness. † † footnotetext: * These authors contributed equally to this work. †These authors jointly supervised this work. 1 Introduction Tables serve as a vital format for representing structured knowledge, yet enabling open-domain question answering (QA) systems to retrieve and interpret tabular information effectively remains a non-trivial challenge. Unlike traditional Tabular QA (TQA) settings that assume access to gold tables, open-domain QA must first identify relevant tables from a large corpus before reasoning over their structure to derive accurate answers Liang et al. 2024 . Simple approaches that linearize tables into text and apply standard text-retrieval methods can lose the syntactic and structural information encoded in rows and columns Badaro et al. 2023 ; Min et al. 2024 . As a result, effective table retrieval has been identified as a key bottleneck for scalable QA over tabular data Li et al. 2025 . Figure 1: Overview of the CRAFT Framework. Recent work has started to tackle open-domain TQA by developing specialized retrieval models. Early approaches adopted sparse or dense text retrievers on flattened tables (e.g., BM25 or bi-encoder models) Fan et al. 2024 ; Robertson and Zaragoza 2009 . For example, the Dense Table Retriever (DTR) introduced by Herzig et al. 2021 augments a bi-encoder with row and column features to better capture table structure. More recent methods explicitly incorporate fine-grained structure: Jin et al. 2023 propose a syntax- and structure-aware retriever that represents each question phrase and each table header and values separately, achieving state-of-the-art results on the NQ-Tables benchmark. Likewise, Li et al. 2025 introduces THYME, a field-aware hybrid retriever that tailors matching strategies to different table fields (e.g., cell values vs. captions). THYME significantly outperforms prior baselines on open-domain table retrieval, highlighting the gains from table-specific modeling. Similarly, Pan et al. 2022 introduced T-RAG, an end-to-end table QA framework that combines table retrievers with sequence-to-sequence generators for retrieval-augmented generation. These advances, however, often rely on complex architectures or extensive fine-tuning on the TQA dataset. This raises an intriguing question: Can off-the-shelf pretrained models, when integrated into a carefully designed retrieval pipeline, achieve competitive performance to expensive domain-specialized, fine-tuned methods? To this end, we propose CRAFT , a modular and scalable multi-stage retrieval framework for TQA that harnesses pretrained models without relying on dataset-specific fine-tuning. Our work introduces CRAFT , a training free cascaded tabular retriever algorithm that achieves competitive performance with state-of-the-art tabular retrievers requiring extensive in-domain training. Through comprehensive ablation studies, we demonstrate the robustness of CRAFT across varying retrieval threshold K , highlighting its adaptability and stability. Building on this foundation, we develop a complete end-to-end TQA system that outperforms all existing TQA- in-the-wild , showcasing the effectiveness of our retrieval approach in real-world, open-domain settings. 2 Methodology To create a retrieval setup without dataset-specific fine-tuning, we adopt a modular framework that leverages off-the-shelf models with multi-stage retrieval, with more complex models for each successive stage. As shown in Figure 1 , CRAFT involves prepossessing followed by stage-wise table retrieval using increasingly expressive models, culminating in answer generation via a pretrained LLM. Our approach demonstrates that pretrained models can effectively retrieve relevant tabular data and generate accurate answers compared to the state-of-the-art framework Zhao et al. 2024 . The pipeline consists of leveraging preprocessing followed by cascaded table retrieval and end-to-end answer generation to maximize semantic alignment between the input queries and tabular content using off-the-shelf embeddings models. 2.1 Preprocessing The preprocessing step involved three key operations to enhance the query and table representations for effective retrieval. Firstly, Gemini-1.5-Flash Team et al. 2025 was employed to generate sub-questions derived from each original query, thus decomposing complex queries into more granular components. Secondly, table titles and comprehensive descriptions were generated using Gemini-1.5-Flash, enriching contextual understanding and enabling more precise downstream retrieval. Finally, all rows across every table were ranked using a Sentence Transformer ( all-mpnet-base-v2 ) model SentenceTransformers 2022 , applied directly to the original queries to assess semantic relevance. The NQ-Tables Dataset statistics are shown in Table 7 . 2.2 Table Retrieval The table retrieval step involves a rigorous, three-stage filtering methodology to identify highly relevant tables for each query. In the first stage, SPLADE Formal et al. 2021 was applied to a comprehensive data set consisting of 169,898 tables, leveraging generated table titles, column headers, individual cell values, and detailed table descriptions to filter and narrow down the candidate tables to the top 5,000 entries. SPLADE leverages sparse lexical expansion to efficiently score candidate passages using minimal memory, making it well-suited for handling large-scale retrieval tasks. In the second stage, the Sentence Transformer ( all-mpnet-base-v2 ) provides a fine-grained semantic refinement by representing each table with mini-tables created through appending column headers to each of the top 5 rows. This refinement approach allows the selection of the top 1,000 mini-tables. In the final stage, the text-embedding-3-small OpenAI 2024 model performs a detailed re-ranking on the top 100 mini-tables, carefully assessing these mini-tables based on their semantic embeddings. Concurrently, we enrich all 919 unique queries using sub-question decomposition generated by Gemini-1.5-Flash to maximize their semantic alignment with the selected tables. 2.3 End-to-End Answer Generation For the end-to-end table question-answering, we use the final retrieval result on the set of 919 queries with three different pretrained models: Llama3-8B , Qwen2.5-7B , and Mistralv0.2-7B . Each LLM generates answers from prompts that include a varying number of tables ( n=\{1,3,5,8,10\} ). Each prompt contains the table title, column headers, and mini-tables (top 5 rows) for n tables, offering comprehensive yet concise context to support accurate answer generation. In addition, we also experiment with the inclusion of a few-shot prompt to guide the text generation process. We evaluate model performance and accuracy by computing the F1 score, which quantifies the overlap between generated answers and reference answers, enabling comparative analysis across different models and table settings. Model R@1 R@10 R@50 Sparse Retrieval Models BM25 18.49 36.94 52.61 SPLADE 39.84 83.33 94.65 Dense Retrieval Models BIBERT 43.78 82.25 93.71 DPR 45.32 85.84 95.44 TAPAS 43.79 83.49 95.10 DTR 32.62 75.86 89.77 SSDR(im) 45.47 84.00 95.05 T-RAG* 46.07 85.40 95.03 Hybrid Retrieval Models BIBERT+BM25 35.87 79.63 94.56 DHR 43.67 84.65 95.62 BIBERT+SPLADE 45.62 86.72 95.62 THYME 48.55 86.38 96.08 CRAFT Stage-wise Stage 1 34.38 72.90 91.62 + Stage 2 36.65 82.91 96.08