ExeSQL : Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects
Jipeng Zhang 1, Haolin Yang 1, Kehao Miao 1, Ruiyuan Zhang, Renjie Pi, Jiahui Gao, Xiaofang Zhou · 2025
Read ExeSQL : Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects by Jipeng Zhang 1, Haolin Yang 1, Kehao Miao 1, Ruiyuan Zhang, Renjie Pi, Jiahui Gao, Xiaofang Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17231.txt.
Introduction
Jipeng Zhang and six co-authors wrote a 2025 paper on ExeSQL, a self taught system that turns plain questions into SQL queries across database dialects. Text to SQL models shine on SQLite thanks to its datasets, but real work spans many dialects with different syntax. Training data for those dialects is scarce, and unchecked generated data is noisy.
ExeSQL bootstraps its own training data through execution. The model drafts queries, runs them, and keeps only those that execute sensibly, a filter called rejection sampling. Preference training on the survivors teaches dialect correct habits. Repeating the loop lets the model adapt to new dialects with verifiable feedback rather than surface filtering. Grounding in real execution is the core idea.
Catalog record
| CREATOR | Jipeng Zhang 1, Haolin Yang 1, Kehao Miao 1, Ruiyuan Zhang, Renjie Pi, Jiahui Gao, Xiaofang Zhou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17231
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ExeSQL : Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects Jipeng Zhang 1, Haolin Yang 1, Kehao Miao 1, Ruiyuan Zhang, Renjie Pi, Jiahui Gao, Xiaofang Zhou Abstract Recent text-to-SQL models have achieved strong performance, but their effectiveness remains largely confined to SQLite due to dataset limitations. However, real-world applications require SQL generation across multiple dialects with varying syntax and specialized features, which remains a challenge for current models. The main obstacle in building a dialect-aware model lies in acquiring high-quality dialect-specific data. Data generated purely through static prompting—without validating SQLs via execution—tends to be noisy and unreliable. Moreover, the lack of real execution environments in the training loop prevents models from grounding their predictions in executable semantics, limiting generalization despite surface-level improvements from data filtering. This work introduces ExeSQL, a text-to-SQL framework with execution-driven, agentic bootstrapping. The method consists of iterative query generation, execution-based filtering (e.g., rejection sampling), and preference-based training, enabling the model to adapt to new SQL dialects through verifiable, feedback-guided learning. Experiments show that ExeSQL bridges the dialect gap in text-to-SQL, achieving average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on PostgreSQL, MySQL, and Oracle, respectively, across multiple datasets of varying difficulty. ExeSQL 1 Introduction With the rapid advancement of Large Language Models (LLMs), their capabilities in assisting with various coding tasks have significantly improved. Tools like GitHub Copilot ( Microsoft, 2023 ; Services, 2023 ) and models such as OpenAI Codex ( Chen et al., 2021a ) have enhanced developer productivity by automating repetitive tasks, providing real-time suggestions, and offering detailed explanations of code functionality. One crucial application of LLMs in software development is the automatic generation of SQL queries from text (text-to-SQL), a task that has gained increasing attention ( Zhong et al., 2017 ; Yu et al., 2018 ; Li et al., 2024a ; Lei et al., 2024 ) . However, most existing research ( Li et al., 2024b ; Zhuang et al., 2024 ; Dong et al., 2023a ; Pourreza and Rafiei, 2024 ; Wang et al., 2023a ; Gan et al., 2021 ; Deng et al., 2021 ) and datasets in the text-to-SQL domain are primarily designed for SQLite, with limited coverage of widely used database systems such as MySQL, PostgreSQL, BigQuery, Oracle, and DuckDB. We incorporate an example of a question with dialect SQL in Figure 1 . The lack of high-quality, dialect-specific text-to-SQL data presents significant challenges in developing models that can generalize across different SQL dialects, ultimately hindering the creation of robust and adaptable text-to-SQL solutions for real-world applications ( Lei et al., 2024 ; Li et al., 2024a ; Pourreza et al., 2024 ) . Rule-Based Translation is Insufficient. Rule-based translation offers a deterministic but rigid solution to SQL dialect conversion. While transpilers like SQLGlot ( Mao, 2023 ) provide structured mappings between dialects, they struggle with complex syntax, schema constraints, and dialect-specific functions ( Zmigrod et al., 2024 ) . Moreover, these systems lack generalizability, require dialect-specific rules ( Li et al., 2024a ; Lei et al., 2024 ) , and cannot guarantee accurate translation. In practice, they still rely on execution-time feedback to detect and fix failures. Maintaining such rule sets is costly and brittle. Even with carefully crafted rules, such systems cannot guarantee perfect accuracy—particularly for complex or edge-case queries—and often rely on execution-time feedback for correction. We provide a detailed analysis in the Appendix A.10 . Figure 1: Given a natural language question, different SQL dialects require distinct syntax adjustments, such as explicit type casting in PostgreSQL. Beyond the traditional text-input–SQL-output formulation, we incorporate the database environment to enable agentic execution feedback for data synthesis and training. Existing Data Collection and Training Lacks Execution Verification. General LLM-based code data generation methods Wei et al. (2023) ; Wang et al. (2022) often fail to account for the specific requirements of text-to-SQL tasks, leading to the creation of syntactically plausible but incorrect SQL queries. These approaches typically generate large amounts of unverified data, which hinders their usefulness for training reliable models. Since SQL outputs can be directly validated through execution, a more structured approach that incorporates execution-based verification and targeted rejection sampling strategies is necessary. Besides, we argue that standard supervised fine-tuning (SFT) alone is insufficient to fully exploit the potential of execution validation, as it does not inherently enforce correctness across dialects. To advance dialect text-to-SQL, we emphasize the importance of both high-quality, executable (text, SQL) data and a training pipeline that directly interacts with the execution environment. We propose an agentic data generation loop that combines LLM-based generation, execution-time validation, and self-correction. This offline loop yields reliable training signals, which are distilled into a dialect-aware model through supervised fine-tuning and offline reinforcement learning. The overall workflow includes: (a) SFT Data Bootstrapping via LLM-based Translation: To mitigate the sparsity of dialect text-to-SQL data and enable effective cold-start training, we leverage high-resource SQLite (text, SQL) pairs and LLMs to efficiently sample dialect SQL queries. This bootstrapped dataset serves as a cold-start fine-tuning set, enabling rapid adaptation to low-resource dialects while minimizing manual annotation. (b) Iterative SFT Data Generation via Execution-based Rejection Sampling: We extend the dataset via an iterative generation–execution–filtering loop, where the model proposes dialect SQLs executed in real databases. Valid outputs are retained through execution-aware rejection sampling, with best-of-N selection enhancing reliability. This agentic cycle uses execution feedback to govern data collection, producing higher-quality training signals without manual effort. (c) Preference Collection via Execution Feedback Rejection Sampling: To further incorporate execution feedback, we distinguish failure types and extract preference pairs—valid versus invalid SQLs—based on their execution results. These are used to train the model with DPO, which guides learning toward executable outputs. This procedure aligns with offline reinforcement learning, leveraging historical execution trajectories to improve model behavior. We summarize our contributions as follows: • We propose an agentic data generation loop that combines LLM-based SQL generation, execution-aware rejection sampling, and iterative self-refinement to construct high-quality dialect-specific training data with minimal manual labeling. • We introduce an offline reinforcement learning framework that captures execution-based preference signals and applies DPO to align the model toward generating executable SQL. • We conduct extensive evaluations across diverse SQL dialects (PostgreSQL, MySQL, and Oracle) \times difficulty levels (single domain, cross-domain, extensive database), demonstrating significant improvements over strong baselines (e.g., GPT-4o) and providing insights for execution-guided SQL modeling. 2 Related Work 2.1 Text-to-SQL Relational databases store a significant portion of the world’s data, and retrieving information from them typically requires writing SQL queries. Automating SQL generation can lower the barrier for users to access data. A common scenario for automatic SQL generation is querying databases using natural language input ( Zhong et al., 2017 ; Yu et al., 2018 ) . Early research treated text-to-SQL as a semantic parsing problem, where models such as RNNs and transformer-based encoders (e.g., BERT) were trained to map natural language questions to SQL statements ( Gan et al., 2021 ; Zhong et al., 2017 ; Deng et al., 2022a ) . Performance has also improved by incorporating additional constraints into inputs and outputs ( Liu et al., 2022 ; Wang et al., 2021 ; Deng et al., 2021 ) . With the emergence of large language models (LLMs) ( Brown et