Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale

Chinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor · 2026

Read Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale by Chinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24023.txt.

Introduction

Soni and co-authors build a small self-hosted model that answers cricket statistics questions in SQL. Putting full schemas into prompts for API models costs much and adds latency. Their two-phase fine-tuning teaches an 8B model the whole database schema. Inputs then shrink by over 99 percent, from 17,000 tokens to under 100. The system serves the CriQ app, a sister app to Dream11. It reaches 98.4 percent execution success and 92.5 percent semantic accuracy, beating a Gemini Flash 2.0 baseline.

Catalog record

CREATORChinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28055
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24023

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale Chinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor Abstract Applying large, proprietary API-based language models to text-to-SQL tasks poses a significant industry challenge: reliance on massive, schema-heavy prompts results in prohibitive per-token API costs and high latency, hindering scalable production deployment. We present a specialized, self-hosted 8B-parameter model designed for a conversational bot in CriQ, a sister app to Dream11—India’s largest fantasy sports platform with over 250 million users—that answers user queries about cricket statistics. Our novel two-phase supervised fine-tuning approach enables the model to internalize the entire database schema, eliminating the need for long-context prompts. This reduces input tokens by over 99%, from a 17k-token baseline to fewer than 100, and replaces costly external API calls with efficient local inference. The resulting system achieves 98.4% execution success and 92.5% semantic accuracy , substantially outperforming a prompt-engineered baseline using Google’s Gemini Flash 2.0 (95.6% execution, 89.4% semantic accuracy). These results demonstrate a practical path toward high-precision, low-latency text-to-SQL applications using domain-specialized, self-hosted language models in large-scale production environments. 1 Introduction The promise of democratizing data access, moving beyond expert-driven analytics to intuitive self-service interfaces, is a key goal for modern data-intensive applications. As user expectations shift towards seamless, conversational interactions, the pressure to deliver performant and intelligent data access tools has intensified. Our team confronted these challenges directly while developing a new feature for our sports intelligence application: a conversational AI assistant that could effortlessly query our complex cricket database, answering any statistical question in plain English (Figure 1 ). Figure 1: Conceptual overview of the conversational AI feature. A user asks a question in natural language, and the system returns a data-driven answer by generating and executing a SQL query. A dominant industry paradigm for enabling this is using powerful, general-purpose Large Language Models (LLMs) via third-party APIs. This approach typically involves feeding the external model a massive prompt containing the user’s question along with the full database schema and a handful of examples (Figure 2 , left). However, this prompt-heavy approach creates significant, often, insurmountable barriers to production deployment. Stuffing the context window with thousands of tokens for every user query is computationally wasteful, leading to prohibitive operational costs (as most API providers charge per token) and high user-facing latency ( 9 ) . This architectural constraint resulted in an unacceptably sluggish user experience, with query latency frequently exceeding several seconds, leading to increased query abandonment. Furthermore, the initial prototype suffered from an inconsistent success rate of only 82% (execution accuracy of generated SQL). These issues highlighted the need for a more cost-effective and reliable solution. Figure 2: A visual comparison of the two paradigms : Left side : “The External API Paradigm”, where a large prompt is sent to a third-party, pay-per-token service. Right side : “The Internalized Paradigm”, showing a small prompt with just the question going to a fine-tuned, self-hosted model with the schema “baked in”. Our initial prototype, leveraging a third-party LLM API, specifically Gemini Flash ( 8 ) , starkly illustrated the profound limitations of off-the-shelf solutions for our critical requirements. Despite Gemini Flash’s well-earned reputation as a leader in low-latency inference, the complex demands of our queries, often requiring a substantial information and cricket context encoded in prompt to encapsulate the necessary context and schema, inherently bottlenecked performance. These critical drawbacks necessitated a fundamental re-architecture. We decisively pivoted from this generic, third-party API approach to develop a specialized, in-house asset: a highly-optimized, fine-tuned model trained using a novel two-phase methodology (Figure 2 , right). This work addresses the critical trade-off between accuracy, cost, and latency. We build upon the concept of schema internalization via fine-tuning, an idea explored in works like YORO ( 25 ; 11 ) , which demonstrated a model’s ability to learn a database structure. Our primary contribution is a novel and practical two-phase fine-tuning recipe specifically designed for production environments. While prior work established the possibility of internalization, our method provides a concrete, structured curriculum that proved essential for enabling a model to reliably recall and apply schema knowledge from a minimal prompt. This approach creates a smaller, low-latency model that beats the accuracy of a large-context system without the associated costs, making it ideal for user-facing products. Our contributions are: • A Novel Two-Phase Fine-Tuning Technique for Schema Internalization: We introduce and validate a new supervised fine-tuning recipe for text-to-SQL that achieves 98.4% execution accuracy while reducing prompt tokens by over 99%. Our method, which explicitly teaches a model to both learn the SQL generation task in full context and to recall the schema from memory, is our core technical contribution. • A Comprehensive Case Study on Productionizing Text-to-SQL: We outline the engineering journey of enhancing our Statsbot, from a limited prompt-based prototype to a deployed, fine-tuned model. This provides a blueprint for balancing accuracy, latency, and cost in user-facing text-to-SQL applications. 2 Related Work Early and Deep Learning Approaches. Initial text-to-SQL research relied on rule-based and statistical methods, as exemplified by early work like ( 6 ) . The advent of deep learning brought sequence-to-sequence (seq2seq) models, with Seq2SQL ( 35 ) being a notable early example. Addressing challenges such as the ”lexical gap” (where natural language queries use different terminology than the database schema), subsequent approaches introduced innovations like SQLNet’s slot-filling mechanism ( 32 ) and the use of intermediate representations such as NatSQL ( 7 ) . A significant breakthrough was RAT-SQL ( 29 ) , which introduced relation-aware self-attention, greatly improving performance by explicitly modeling the relationships between query tokens and database schema elements. Further advancements included semi-autoregressive parsing ( 24 ) , grammar-augmented pre-training ( 33 ) , and methods that decoupled the text-to-SQL task into sub-problems ( 15 ; 17 ) . This progress was significantly accelerated by the development of challenging benchmark datasets, including WikiSQL ( 36 ) , Spider ( 34 ) , KaggleDBQA ( 13 ) , BIRD ( 18 ) , and Dr.Spider ( 4 ) . The Rise of Large Language Models (LLMs). The introduction of transformer-based LLMs, such as T5 ( 22 ) and BART ( 14 ) , fundamentally reframed text-to-SQL as a text-to-text generation problem. This paradigm shift was further propelled by models like GPT-3 ( 3 ) , which popularized in-context learning (ICL). ICL-based systems harness the powerful reasoning capabilities of LLMs through elaborate prompts. Techniques like Chain-of-Thought (CoT) prompting ( 31 ; 26 ) and self-correction mechanisms ( 5 ) , exemplified by DIN-SQL ( 20 ) , have achieved state-of-the-art results in text-to-SQL. However, these ICL-heavy approaches, which often necessitate extensive prompt engineering, come with significant drawbacks, including prohibitive latency and cost, making them impractical for real-time industrial applications. For instance, DIN-SQL with GPT-4 has reported a 60-second latency and a cost of $0.50 per query ( 20 ) . This limitation has spurred the development of capable open-source models like LLaMA ( 27 ; 28 ) , specialized code generation models such as StarCoder ( 19 ) and CodeLlama ( 23 ) , and SQL-centric pre-trained models like the CODES series ( 16 ) . Schema Internalization and PEFT. Our work directly addresses the performance bottleneck

Read the complete text