TableCopilot : A Table Assistant Empowered by Natural Language Conditional Table Discovery

Lingxi Cui, Guanyu Jiang, Huan Li, Ke Chen, Lidan Shou, Gang Chen · 2025

Read TableCopilot : A Table Assistant Empowered by Natural Language Conditional Table Discovery by Lingxi Cui, Guanyu Jiang, Huan Li, Ke Chen, Lidan Shou, Gang Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08283.txt.

Introduction

Lingxi Cui and five co-authors build a table assistant that finds the table you need before analyzing it, in a 2025 paper. Most assistants assume the user already has the right table. In huge table pools, finding it is half the work.

The new task lets users give a plain language condition plus a sample table. TableCopilot then searches for matching tables and works with them. Its engine, Crofuma, blends matches from each input type alone and combined. In tests it beat the best single input methods by at least 12 percent on a ranking score. Code, data, and a demo video are public.

Catalog record

CREATORLingxi Cui, Guanyu Jiang, Huan Li, Ke Chen, Lidan Shou, Gang Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08283

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TableCopilot : A Table Assistant Empowered by Natural Language Conditional Table Discovery Lingxi Cui, Guanyu Jiang, Huan Li, Ke Chen, Lidan Shou, Gang Chen Abstract . The rise of LLM has enabled natural language-based table assistants, but existing systems assume users already have a well-formed table, neglecting the challenge of table discovery in large-scale table pools. To address this, we introduce TableCopilot , an LLM-powered assistant for interactive, precise, and personalized table discovery and analysis. We define a novel scenario, nlcTD , where users provide both a natural language condition and a query table, enabling intuitive and flexible table discovery for users of all expertise levels. To handle this, we propose Crofuma , a cross-fusion-based approach that learns and aggregates single-modal and cross-modal matching scores. Experimental results show Crofuma outperforms SOTA single-input methods by at least 12% on NDCG@5. We also release an instructional video, codebase, datasets, and other resources on GitHub to encourage community contributions. TableCopilot sets a new standard for interactive table assistants, making advanced table discovery accessible and integrated. † † The definitive Version of Record was published in VLDB’25, https://dl.acm.org/doi/10.1145/xxxx . 1. Introduction The rise of large language models (LLMs) has extended their capabilities to tabular data, enabling the development of table assistants for tasks such as TableQA ( Zhu et al. 2024 ) , tabular data manipulation ( Zhang et al. 2024 ) and analytics ( Wang et al. 2025 ) . These approaches have achieved notable success in downstream tasks, often assuming that users already have a table with sufficient data to work with. However, in many real-world scenarios, users may start without a table or need to enhance an existing one before the analysis begins ( Cui et al. 2024 ) . This introduces the need for an additional, often overlooked step: table discovery over large-scale table pools, which has received limited attention in the context of table assistants. To empower table assistants with table discovery capabilities, we face two primary challenges. (1) LLM Incompatibility with Table Discovery : The foundation of table assistants, LLMs, are not inherently optimized for table discovery tasks. Importing a large-scale pool with millions of tables into an LLM is impractical due to token limits and privacy concerns. Moreover, LLMs trained primarily on sequential textual data struggle to interpret structured, tabular data, which are of two-dimensional nature. This necessitates the use of specialized table discovery methods. (2) Gaps in Existing Table Discovery Methods : Current methods typically rely on either keyword(s) or a table as a query to retrieve relevant tables. However, keywords and single-table queries often fail to fully capture user intent, particularly in interactive table assistants where users may specify detailed natural language requirements (see Example 1.1 ). Traditional methods focusing solely on keywords or query tables are therefore insufficient for such use cases. Example 0. Suppose a recruiter wants to populate a shortlist table with applicants from an application database for subsequent competence evaluation. This shortlist table includes fields such as ID, name, graduation year, CS grade, math grade, and phone number. The recruiter is looking for records that can be directly filled into this table, with a specific request: applicants must graduate in 2023 or 2024, and have a CS grade above 90. In this context, keyword-based table discovery methods cannot handle such complex and personalized NL requirements, and the retrieved tables cannot be directly unioned with the shortlist. Moreover, query-table-based discovery methods are even less applicable, as the retrieved tables cannot meet the NL requirements. Therefore, a table assistant with discovery capabilities is the ideal solution. It is designed for users unfamiliar with database structures, meets complex and personalized needs, and facilitates further analysis and processing of data. To empower LLM-based table assistants with table discovery capabilities, we make the following efforts: (1) New User Scenario. We introduce a new user scenario: Natural Language Conditional Table Discovery ( nlcTD ), allowing users to provide both natural language (NL) requests and a query table. Based on practical application needs, we define three key user cases within nlcTD : NL-only search that takes only NL as input; NL-conditional table union search that targets row-based table unions, and NL-conditional table join search that focuses on column-based table joins. By incorporating NL conditions into nlcTD , we bridge the gap between LLM-based table assistants and table discovery. This integration enables interactive, precise, and personalized searches, enhancing user experience. Moreover, detailed NL conditions facilitate highly efficient searches across vast table repositories, reducing LLM calls and improving response speed, scalability, and cost-effectiveness. (2) Cross-Fusion-Based Solution. To address the challenges of this new practical scenario, where existing table discovery methods fail to simultaneously process query tables and NL conditions, we propose Crofuma , a novel cross-fusion-based matching method for table discovery. Crofuma calculates single-modal table matching scores and cross-modal NL condition matching scores separately, then aggregates them during online query processing (Section 2.3 ). Crofuma can handle situations with only NL or a single table as input, making it compatible with traditional table discovery scenarios. In the offline phase, we employ pretrained language models (PLMs) as encoders for both NL and tables. To better capture table structure and semantics, we enhance PLMs through contrastive learning during pretraining. Table contents and metadata are encoded separately and indexed using HNSW ( Fan et al. 2023 ; Dong et al. 2023 ) , achieving an average query time of under 500 ms and enabling real-time interactions with LLMs (Section 2.2 ). Experimental results show that our Crofuma outperforms SOTA single-input discovery methods by at least 12% on NDCG@5 in the nlcTD scenario (Section 2.4 ). (3) End-to-end Prototype. We present TableCopilot , a fully functional prototype that seamlessly integrates Crofuma into an LLM-powered table assistant. TableCopilot supports the entire workflow from NL-conditional table discovery to downstream applications like TableQA. To ensure robustness and facilitate evaluation, TableCopilot also incorporates existing table discovery methods for direct comparison (search panel in Figure 2 (a)). This design demonstrates the practicality and superiority of equipping LLM-based assistants with native, efficient table discovery capabilities. Our proposals offer significant technical advancements in two key areas: (a) LLM-based Table Assistants . Recent studies have explored leveraging LLMs to enhance table-related tasks: Zhu et al. ( Zhu et al. 2024 ) leverage multi-agent LLMs for TableQA. iDataLake ( Wang et al. 2025 ) integrates LLMs into analytics systems for querying unstructured, semi-structured, and structured data in data lakes. Zhang et al. ( Zhang et al. 2024 ) present TableLLM , an LLM-driven assistant capable of advanced operations like querying, updating, merging, and charting. While these works excel in table analysis and manipulation, they can benefit from our solution, which introduces natural language conditional table discovery to enable seamless and comprehensive end-to-end table workflows. (b) Table Discovery . Existing table discovery prototypes primarily focus on keyword-based or query-table-based search: Aurum ( Castro Fernandez et al. 2018 ) retrieves related tables by performing schema-based matching on user-provided keywords. Auctus ( Castelo et al. 2021 ) supports joinable and unionable table searches using query tables. LakeCompass ( Chai et al. 2024 ) combines keyword and query-table-based table search while considering follow-up ML task support. However, these prototypes do not address the new nlcTD scenario that

Read the complete text