DBRouting: Routing End User Queries to Databases for Answerability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Top-k retrieval in databases
Priyangshu Mandal, Manasi Patwardhan, Mayur Patidar, Lovekesh Vig · 2025
Read DBRouting: Routing End User Queries to Databases for Answerability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Top-k retrieval in databases by Priyangshu Mandal, Manasi Patwardhan, Mayur Patidar, Lovekesh Vig (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16220.txt.
Introduction
Mandal, Patwardhan, Patidar, and Vig define in twenty twenty-five a new task: sending a user's plain-language question to the right company database. Large firms store data across many sources, and finding the source that can answer a question is a basic hurdle. Their task assumes each source is a structured database with its own topic and tables.
They built test datasets by extending sets made for turning language into database queries. They tried both large language models and vector-based methods, with and without task training. Language models scored higher but struggled with long inputs. Trained vector methods gained most when sample questions per database were plentiful. More sources, close topics, and vague questions all made the task harder.
Catalog record
| CREATOR | Priyangshu Mandal, Manasi Patwardhan, Mayur Patidar, Lovekesh Vig |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16220
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DBRouting: Routing End User Queries to Databases for Answerability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Top-k retrieval in databases CCS: Information systems Language models Priyangshu Mandal, Manasi Patwardhan, Mayur Patidar, Lovekesh Vig Abstract . Enterprise level data is often distributed across multiple sources and identifying the correct set-of data-sources with relevant information for a knowledge request is a fundamental challenge. In this work, we define the novel task of routing an end-user query to the appropriate data-source, where the data-sources are databases. We synthesize datasets by extending existing datasets designed for NL-to-SQL semantic parsing. We create baselines on these datasets by using open-source LLMs, using both pre-trained and task specific embeddings fine-tuned using the training data. With these baselines we demonstrate that open-source LLMs perform better than embedding based approach, but suffer from token length limitations. Embedding based approaches benefit from task specific fine-tuning, more so when there is availability of data in terms of database specific questions for training. We further find that the task becomes more difficult (i) with an increase in the number of data-sources, (ii) having data-sources closer in terms of their domains,(iii) having databases without external domain knowledge required to interpret its entities and (iv) with ambiguous and complex queries requiring more fine-grained understanding of the data-sources or logical reasoning for routing to an appropriate source. This calls for the need for developing more sophisticated solutions to better address the task. Figure 1. Database (DB) Routing Task 1. Introduction Giant enterprises work across multiple verticals (domains) and the relevant data is distributed across multiple data-sources. These sources can be in various forms such as knowledge graphs, databases, document repositories, etc. As a part of an enterprise-level search, when a query in natural language is fired by an end-user, there is a need to route the query to a set of appropriate sources, which carry the data to correctly answer the query, as shown in fig 1 . Considering this real-life scenario, we define a new task of routing a natural language query to an appropriate data source. We have restricted the data-sources to enterprise databases (DBs) containing information about various domains, which are one of the main types of data-sources in an enterprise setting. There is an existing work on routing of natural language (NL) queries to (i) distinct proprietary and open-source Large Language Models (LLMs) ( Ong et al. 2024 ; Ding et al. 2024 ) to strike a balance between performance and cost, (ii) either Retrieval Augmented Generation (RAG) based or Long Context (LC) LLM based approaches ( Li et al. 2024b ) , (iii) distinct Tools or API calls ( Qin et al. 2023 ; Hao et al. 2023 ; Patil et al. 2023 ; Schick et al. 2023 ) . However, to the best of our knowledge, these approaches have not taken into consideration enterprise DBs as data-sources. Also unlike API documentations, which have clear descriptions about API usage, data-sources such as databases may not have high-level descriptions available for their content. Also, for more fine-grained queries such high-level descriptions may not be sufficient. Hence, the approaches applied for Tool/API routing may not be directly applicable to DB routing. Existing work on question answering over multiple sources mainly take (i) only multiple documents ( Schuster et al. 2023 ; Lee et al. 2024 ; Wang et al. 2023 ) , (ii) multiple knowledge graphs ( Zhang et al. 2023 ) or (iii) multiple heterogeneous sources such as knowledge graphs, tables from web and text documents ( Christmann et al. 2023a ; Zhang et al. 2024 ; Lehmann et al. 2024 ; Zhao et al. 2023 ; Christmann et al. 2023b ; Sun et al. 2018 ) into consideration. However, to the best of our knowledge, we are the first ones to define the task of routing queries posed on multiple enterprise DBs. As opposed to other forms of data-sources such as knowledge graphs, text documents, etc, databases have some unique characteristics in that they consist of a combination of structured as well as unstructured data represented in the form of database schema and the associated meta-data or business rules. Moreover, in an enterprise setting multiple databases may contain information from various domains with some domain overlap leading to overlap between DB entities (table names, column names and values), making the routing task even more difficult. In the future, we plan to extend our problem by taking into consideration other data-sources along with DBs for the task of routing and question-answering over heterogeneous data-sources, which would be closer to the realistic scenario of an enterprise search. Due to the unavailability of existing datasets to benchmark for this novel DB routing task, we extend existing datasets designed for NL-to-SQL semantic parsing in the cross-domain setting ( Yu et al. 2018 ; Li et al. 2024a ) . We apply existing embedding and open-source Large Language Model based approaches to benchmark these datasets and also throw some light on the challenges with these approaches. We try to answer the following Research Questions (RQs): (1) Does domain specific (availability of questions from the same database) or cross-domain training assist to improve the performance of the DB-routing task? (2) Does similarity in domains of the data-sources affect the performance of the DB-Routing task? (3) Does an increase in the number of data-sources affect the performance of the DB Routing Task? (4) Does domain specific external knowledge facilitate in improving the performance of the DB-Routing task? The main contributions of this work are as follows: • We are the first ones to define the Database (DB) routing task for routing NL queries to the correct DB. • We extend two existing datasets, viz, Spider ( Yu et al. 2018 ) and BirdSQL ( Li et al. 2024a ) , designed for cross-domain semantic parsing task for DB-Routing. • We create benchmarks on these datasets using state-of-the-art embedding, task specific embedding and open-source Large Language Model (LLM) based approaches. • We elaborate on the challenges faced by these approaches for the task, which calls for the need for a more sophisticated solution for the task. 2. Problem Statement We have an end-user question q and a set of databases D , where each database is indexed by a database id d_{x} with schema S_{x} consisting of multiple tables and columns ( T_{x} , C_{xy} ), where T_{x} are tables that belong to database schema S_{x} and C_{xy} are the columns of table T_{x} . The task is to rank the databases in D for the question q based on their relevance to the question in terms of answerability (Database can provide correct answer to the question). We assume to have training data Tr=\{Q_{tr},D_{tr}\} , which provides a mapping for each question q_{tr}\in Q_{Tr} to a database d_{tr}\in D_{Tr} . In our current setting, we assume every question has a mapping to atleast one database (all questions are answerable) and is mapped to ONLY ONE database. We assume to have two types of test sets. In-domain test set Te_{in} consists of a set of questions Q_{in} and set of