Meta-aware Learning in text-to-SQL Large Language Model*

Wenda Zhang · 2025

Read Meta-aware Learning in text-to-SQL Large Language Model* by Wenda Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18929.txt.

Introduction

Wenda Zhang improves how language models write database queries in a 2025 paper. Turning plain questions into SQL is hard where domain terms and table layouts grow complex. Business users need right answers, not near misses.

Zhang's frame teaches the model on four fronts at once: table layout, step-by-step reasoning traces, domain facts, and links between data fields. Fine-tuning on all four gives the model a fuller grasp of each database. Two test rounds showed better run accuracy and wider task range. The frame also cut the loss of old skills when new ones were taught. The work targets real business data work.

Catalog record

CREATORWenda Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18929

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Meta-aware Learning in text-to-SQL Large Language Model* Wenda Zhang Abstract The advancements of Large language models (LLMs) have provided great opportunities to text-to-SQL tasks to overcome the main challenges to understand complex domain information and complex database structures in business applications. In this paper, we propose a meta-aware learning framework to integrate domain knowledge, database schema, chain-of-thought reasoning processes, and metadata relationships to improve the SQL generation quality. The proposed framework includes four learning strategies: schema-based learning, Chain-of-Thought (CoT) learning, knowledge-enhanced learning, and key information tokenization. This approach provides a comprehensive understanding of database structure and metadata information towards LLM through fine-tuning to improve its performance on SQL generation within business domains. Through two experimental studies, we have demonstrated the superiority of the proposed methods in execution accuracy, multi-task SQL generation capability, and reduction of catastrophic forgetting. Keywords: text-to-SQL LLM, fine-tuning, meta-aware leanring, metadata, chain-of-thought, BigQuery SQL, business database. I INTRODUCTION As one of the most important tasks in natural language processing (NLP) to translate plain human language into structured query language (SQL), text-to-SQL enables non-experts to interact effectively with data in the increasingly complex business database environments [ 1 ] . The recent advancements of large language models (LLMs) have presented great opportunities to the progress of text-to-SQL models. The LLM-based text-to-SQL approaches including in-context learning (ICL) [ 2 ] and fine-tuning (FT) [ 3 ] offer significant advantages over traditional rule-based approaches [ 4 , 5 ] and neural network based approaches [ 6 , 7 ] by providing flexibility in handling complex databases, improving adaptability to a variety of systems, and saving development efforts in SQL generation tasks. However, translating natural language to SQL in real-world business contexts presents several challenges. Business databases often exhibit complexity, including complex schemas, domain-specific topics and data formats, and platform-specific dialects such as BigQuery SQL. In-context learning with zero-shot methods [ 8 ] on existing text-to-SQL LLMs trained on public SQL databases (d.g., SQLCoder 2 2 2 https://defog.ai/blog/open-sourcing-sqlcoder , DAIL-SQL [ 9 ] and GPT-4 [ 10 ] ) face difficulties in identifying the domain-specific unique complexities and knowledge inherent in business databases. An example of SQL structure is shown in Figure 1 , two similar questions with the topic of ‘‘top 5 object” are listed from a BigQuery public database 3 3 3 https://huggingface.co/datasets/derekiya/BigQuery_dataset-v3 and a business database in BigQuery dialect. The business database uses non-standard date definitions and maintains date information across separate tables. Fig. 1: The comparison in SQL complexity from public database and the business database. In the question from business database, YTD refers to “year to date” and the definition of years differs from the standard calendar year. The date information is maintained across a separate table. The key information is masked to present the SQL structures. Additionally, few-shot learning requires large sets of example queries to derive the desired output. As database complexity increases, the demand for input tokens grows exponentially. This may lead to significantly increased associated costs on proprietary LLMs or encounter limitations in input tokens or devices on open-source LLMs [ 1 ] . These issues can impede LLMs from effectively incorporating the essential business domain-specific information and exacerbate issues related to hallucinations [ 11 ] . Fine-tuning an open-source LLM for specific business environments on text-to-SQL tasks is a commonly adopted solution [ 1 , 3 ] , which can partially mitigate these issues. However, a vanilla fine-tuning [ 12 , 13 ] may not effectively incorporate complex domain knowledge including specialized terms, abbreviations, custom time-frame definitions, and complex table relationships. To address these challenges, we propose a meta-aware learning strategy designed to enable LLMs to incorporate domain knowledge through fine-tuning, optimizing their performance for domain-specific, complex business database environment. By concentrating on multiple aspects in a business domain including table schema details, chain-of-thought reasoning process to generate SQL for a business question, domain knowledge relationships, and key information integration, the proposed method can increase the capability for an LLM to generate accurate SQL queries in a complex business database environment. The paper is structured as follows: Section II presents the related work to this study. Section III outlines the proposed meta-aware learning approach, elaborating the components including schema-based learning, chain-of-thought (CoT) learning, domain knowledge enhancement learning, and key information tokenization method. In section IV , we present experiment results on real business databases at Walmart to evaluate the effectiveness of our approach on multi-table domain-specific tasks, followed by conclusions and future work directions in section V . II RELATED WORK Research on text-to-SQL has been greatly developed in recent decades. The early work of text-to-SQL research was conducted with rule-based approaches, which aimed to parse information with natural language processing (NLP) techniques from input questions to generate SQL queries using predefined templates and syntax rules [ 4 , 5 ] . Recent advancements in deep learning have significantly facilitated the development of syntax parsing by integrating the mapping function from input natural languages to the SQL output [ 14 , 15 , 9 ] . Deep learning methods, including long-short-term memory (LSTM) [ 14 ] , convolutional neural network (CNN) [ 16 ] , sequence-to-sequence (Seq2Seq) [ 6 ] , and the attention mechanism [ 7 ] , have been used in questions and database information encoding to improve context integration and syntax understanding in text-to-SQL tasks. The emergence of large language models (LLMs) has provided a groundbreaking solution to text-to-SQL [ 1 , 9 ] , with significant capability in syntax understanding, fluent output generation, and knowledge storage. Pre-trained LLMs, such as GPT [ 10 ] , with in-context learning (ICL) [ 17 , 18 ] , can effectively synthesize schema information from input prompts to generate accurate SQLs [ 8 , 19 ] . By providing reasoning steps from input questions to formulate the SQL output, the Chain-of-thought (CoT) approach [ 20 , 21 , 22 ] can enhance the performance of SQL generation. Furthermore, retrieval-augmented generation (RAG) [ 23 , 24 ] is a promising technique recently used to enhance text-to-SQL performance by integrating relevant context from external sources and improving understanding of domain-specific queries. Another widely adopted approach to leverage LLM’s capability is to integrate question-SQL mapping information through fine-tuning techniques [ 1 , 3 , 12 , 13 ] . However, the implementation of fine-tuning in text-to-SQL LLMs is prone to issues like catastrophic forgetting [ 25 , 26 ] , where pre-trained knowledge is lost during fine-tuning, low computational efficiency [ 25 ] , and hallucination [ 11 ] , where models generate inaccurate or unrelated SQL queries to the provided schemas and contexts. To address these issues, researchers have explored strategies in LLM fine-tuning for text-to-SQL applications, including parameter-efficient fine-tuning (PEFT) methods [ 3 , 27 ] , domain-specific knowledge incorporation [ 1 ] , and multi-task fine-tuning strategies [ 28 ] , which provided novel perspectives on text-to-SQL solutions. III METHODOLOGY In this paper, we propose a meta-aware learning framework to improve text-to-SQL performance by integrating multiple targeted learning strategies. This framework combines several complementary subprocesses: schema-based learning, chain-of-thought reasoning, domain knowledge enhancement, and key information tokenization. Each component is designed to address specific challenges in text-to-SQL tasks, such as schema comprehension,

Read the complete text