Database Entity Recognition with Data Augmentation and Deep Learning

Zikun Fu, Chen Yang, Kourosh Davoudi, Ken Q. Pu · 2025

Read Database Entity Recognition with Data Augmentation and Deep Learning by Zikun Fu, Chen Yang, Kourosh Davoudi, Ken Q. Pu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19372.txt.

Introduction

Zikun Fu and co-authors tackle entity recognition in plain language questions asked of a database, in a 2025 paper. A system that turns questions into queries must first spot which words name tables, columns, or values. Labeled examples for that spotting task were missing.

The team built a hand labeled benchmark from popular question to query sets. They also grew training data automatically, using the known queries to tag their questions. A specialized model on a T5 backbone then tags the entities. It beat two leading general taggers on precision and recall. Data growth added more than ten points; backbone tuning added several more.

Catalog record

CREATORZikun Fu, Chen Yang, Kourosh Davoudi, Ken Q. Pu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19372

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Database Entity Recognition with Data Augmentation and Deep Learning Zikun Fu, Chen Yang, Kourosh Davoudi, Ken Q. Pu Abstract This paper addresses the challenge of Database Entity Recognition (DB-ER) in Natural Language Queries (NLQ). We present several key contributions to advance this field: (1) a human-annotated benchmark for DB-ER task, derived from popular text-to-sql benchmarks [ 6 , 11 ] , (2) a novel data augmentation procedure that leverages automatic annotation of NLQs based on the corresponding SQL queries which are available in popular text-to-SQL benchmarks, (3) a specialized language model based entity recognition model using T5 as a backbone and two down-stream DB-ER tasks: sequence tagging and token classification for fine-tuning of backend and performing DB-ER respectively. We compared our DB-ER tagger with two state-of-the-art NER taggers, and observed better performance in both precision and recall for our model. The ablation evaluation shows that data augmentation boosts precision and recall by over 10 % 10\% , while fine-tuning of the T5 backbone boosts these metrics by 5 − 10 % 5-10\% . I Introduction Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP) that involves identifying and classifying named entities in text into predefined categories. While traditional NER focuses on general entity types, we address a more specialized challenge: database entity recognition in natural language queries (NLQ). This task is particularly crucial in the context of text-to-SQL systems, where accurately identifying database entities in natural language queries serves as a vital intermediate step. By properly tagging database entities in NLQs, downstream models can more effectively generate and validate complete queries across various query languages. In this paper, we specifically tackle the Database Entity Recognition (DB-ER) problem using deep learning approaches. Our work distinguishes itself from conventional NER systems by leveraging existing text-to-SQL training data, where each NLQ is paired with its corresponding SQL query. This unique pairing provides an opportunity to enhance our understanding of database entities through the structural information present in SQL queries, enabling more sophisticated data augmentation techniques. To facilitate the creation of high-quality training data for our DB-ER model (defined in Section V ), we have developed a web application that supports collaborative annotation of NLQ samples. This tool enables multiple users to concurrently annotate database entities while considering both the underlying database schema and the corresponding SQL query. Figure 2 shows part of the user interface. Given the inherent challenges of manual annotation, including high costs and limited efficiency, we introduce an innovative data augmentation algorithm. This algorithm automatically annotates NLQs by utilizing the structural information from their corresponding SQL queries. This approach significantly expands our training dataset while maintaining annotation quality, providing a scalable solution to the data scarcity problem in DB-ER due to the labour intensive nature of data labeling. Our technical contribution includes a specialized DB-ER tagger built upon the T5 language model architecture. Through extensive experimentation and ablation studies, we have identified two key techniques that substantially improve DB-ER performance: (1) data augmentation using our auto-annotation system, which boosts accuracy by over 10%, and (2) fine-tuning the T5 backbone with a seq2seq learning task, which further improves end-to-end tagging accuracy by 5-10%. These improvements demonstrate the effectiveness of our approach in addressing the challenges of database entity recognition. Figure 1 shows the overall pipeline including data augmentation, training, and evaluation. Fig. 1: Data augmentation pipeline for training and validating DB-ER tagger. In our pipeline, human generated annotations are augmented with auto-annotation. The DB-ER tagger trained on the augmented dataset is then evaluated against the human annotated test dataset, which was held-out in the data augmentation process. II Problem description In this section, we formally define the Database Entity Recognition (DB-ER) problem, and outline a machine learning approach to the problem and data augmentation with synthetic annotation. II-A DB-ER tagging Definition II.1 (NLQ and Spans) . A NLQ x is a sequence of words (or tokens). Given x\in\mathbf{NLQ} , we denote \mathbf{Spans}(x) as all token subsequences of x . Each span s\in\mathbf{Spans}(x) is characterized by the start and end position: (i,j) where 0\leq i\leq j<|x| . In the context of database entity recognition, we need to identify specific spans within a NLQ that correspond to database entities. These entities can be of different types, as defined below. Definition II.2 (Database Entity) . A database entity e=(\mathrm{text}(e),\mathrm{type}(e)) where \mathrm{text}(e) is the text value of the entity as a sequence of tokens, and \mathrm{type}(e) is the type of the entity, which can be one of the following: table names ( \mathbf{T} ), column names ( \mathbf{C} ), cell values ( \mathbf{V} ). We denote the set of all database entities 1 1 1 In this paper, database entities \mathcal{E} refer to the typed string values of table names, column names and database values. This is not to be confused with entities in traditional Entity-Relation (ER) modeling. as \mathcal{E} . Definition II.3 (Annotation) . An annotation h_{x} of x\in\mathbf{NLQ} is a partial mapping from \mathbf{Spans}(x) to \mathcal{E} . Alternatively, we can think of h_{x} as sequence labeling of tokens in x by labels from \mathcal{E}\cup\{\mathrm{none}\} where \mathrm{none} means that the corresponding token is not part of a database entity. We define the space of all possible DB-ER annotations as \mathcal{A}=\{h_{x}\mid x\in\mathbf{NLQ}\} . A DB-ER tagger is a function H:\mathbf{NLQ}\to\mathcal{A} . We focus on a machine learning approach to DB-ER. Namely, we will train a model H_{\theta} as the DB-ER tagger that will take NLQ as input and output a DB-ER annotation h_{x} . The architecture is given by a two-headed deep neural network in which we use T5-encoder has the embedding backbone, and have two separate heads: T5-decoder for sequence tagging, and a linear classifier for per-token classification into \{\mathbf{T},\mathbf{C},\mathbf{V},\mathrm{none}\} . x z y_{\mathrm{seq}} y_{\mathrm{cls}}[i] \mathbf{T5Enc} \mathbf{T5Dec} \mathbf{Classifier} II-B Synthetic annotations The training data for DB-ER comes from text-to-SQL datasets such as Spider [ 11 ] and BIRD [ 6 ] . These datasets contain pairs of natural language queries and their corresponding SQL queries. We leverage this pairing to help prepare training data for DB-ER that optimally matches spans of NLQ with database entities in the corresponding SQL query. The synthetic annotation is a function: \mathbf{syn}:\mathbf{NLQ}\times\mathbf{SQL}\to\mathcal{A} The details of data augmentation is given in Section IV . We combine the ground truth annotations and the automatically generated annotations to create a larger training set for the DB-ER model. Fig. 2: The collaborative annotation interface. III Related work III-A Named entity recognition Traditional NER have achieved significant success by leveraging contextual embeddings and deep learning architectures. LUKE [ 10 ] introduces entity-aware self-attention mechanisms to improve entity representations, while Flair [ 1 ] utilizes contextual string embeddings to capture character-level features. spaCy [ 5 ] offers an efficient pipeline for NLP tasks, including a robust NER model. These models achieve high accuracy in standard benchmarks (e.g., CoNLL-2003), but they are typically trained on open-domain entities (person, organization, location, etc.) and may falter on specialized domains with unseen entity vocabularies. III-B Text-to-SQL and schema linking The task of text-to-SQL parsing has spurred the development of cross-domain benchmarks such as BIRD [ 6 ] and Spider [ 11 ] . A core challenge in text-to-SQL is schema linking [ 2 ] –

Read the complete text