BASE-SQL: A powerful open source Text-To-SQL baseline approach

Lei Sheng, Shuai-Shuai Xu, Wei Xie · 2025

Read BASE-SQL: A powerful open source Text-To-SQL baseline approach by Lei Sheng, Shuai-Shuai Xu, Wei Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10739.txt.

Introduction

Lei Sheng and co-authors present BASE-SQL in a 2025 paper, an open-source pipeline that turns plain questions into SQL database queries. Fine-tuned open models lacked a simple, strong, cheap baseline for this task.

The pipeline runs four steps: schema linking, candidate generation, revision, and merge revision. It fine-tunes a mid-size open code model rather than calling closed giants. Scores beat other open model methods on two standard benchmarks and topped some closed model systems. Each query needs only a handful of model calls on average. Privacy, cost, and speed all favor local open models in real deployments. The code is released for public use.

Catalog record

CREATORLei Sheng, Shuai-Shuai Xu, Wei Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10739

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BASE-SQL: A powerful open source Text-To-SQL baseline approach Lei Sheng, Shuai-Shuai Xu, Wei Xie Abstract The conversion of natural language into SQL language for querying databases (Text-to-SQL) has broad application prospects and has attracted widespread attention. At present, the mainstream Text-to-SQL methods are mainly divided into in-context learning (ICL) based methods and supervised fine-tuning (SFT) based methods. ICL-based methods can achieve relatively good results thanks to the use of the most advanced closed-source models. However, in real-world application scenarios, factors such as data privacy, SQL generation efficiency and cost need to be considered. SFT-based methods have certain advantages. At present, methods based on fine-tuning of open source models lack easy-to-implement and effective (cost-effective) baseline methods. We propose a pipeline-based method using open source model fine-tuning, referred to as BASE-SQL, which includes four components: Schema Linking, Candidate SQL Generate, SQL Revision and SQL Merge Revision. Experimental results show that BASE-SQL uses the open source model Qwen2.5-Coder-32B-Instruct, and achieves an accuracy of 67.47% on the BIRD development set and 88.9% on the Spider test set, which is significantly better than other methods using open source models, and even exceeds several methods using the GPT-4o closed-source model. At the same time, BASE-SQL is easy to implement and highly efficient (on average, only five calls to the large language model are required to generate SQL once). The code will be open sourced at https://github.com/CycloneBoy/base_sql . 1 Introduction Text-to-SQL task is to convert natural language into Structured Query Language (SQL) for querying database, which has received more and more attention ( Qin et al., 2022 ; Katsogiannis-Meimarakis and Koutrika, 2023 ; Liu et al., 2024 ; Shi et al., 2024 ) . It can reduce the difficulty of interaction between users and databases, especially in the fields of data analysis, business intelligence, intelligent customer service and other broad application prospects. With the rapid development of deep learning technology, Text-to-SQL tasks have made rapid progress, from only being able to answer the domain-specific single-table questions before, to being able to solve cross-domain multi-table complex problem SQL generation. On the famous Spider ( Yu et al., 2019 ) benchmark, it has achieved an accuracy rate of 90%. Early Text-to-SQL methods were mainly based on the sequence-to-sequence (Seq2Seq) generation method ( Wang et al., 2020 ; Guo et al., 2019 ; Li et al., 2023a ) . The current mainstream is the method based on large language models (LLMs), which can be roughly divided into in-context learning (ICL) based methods and supervised fine-tuning (SFT) based methods ( Liu et al., 2024 ) . ICL-based methods ( Pourreza and Rafiei, 2023 ; Gao et al., 2023 ; Lee et al., 2024 ; Pourreza et al., 2024 ) mainly use the most advanced closed-source LLMs(such as: GPT-4, GPT-4o, Gemini-1.5 Pro, etc.), and then use various strategies to improve state-of-the-art (SOTA) performance without considering the efficiency and cost of SQL generation. Benefiting from the powerful reasoning ability of the model, it often achieves better results. SFT-based methods use open source LLMs for fine-tuning to improve the effect of SQL generation ( Pourreza and Rafiei, 2024 ; Li et al., 2024a ; Yang et al., 2024b ) . Due to the small size of the model used, it often performs worse than ICL-based methods. In real-world Text-to-SQL application scenarios, the following factors will be considered: data privacy, efficiency and cost of SQL generation. Therefore, the method of SFT-based on open source models has certain advantages. CodeS ( Li et al., 2024a ) uses a specially curated SQL-centric corpus for incremental pre-training, and combines it with a two-way data augmentation technique for SFT, which ultimately exceeds the methods that use a large number of closed-source models, but it relies on a large amount of data for incremental pre-training. SENSE ( Yang et al., 2024b ) uses data synthesized by closed-source models to fine-tune small models, demonstrating the effectiveness of its synthetic data, but it relies on closed-source models for data synthesis. DTS-SQL ( Pourreza and Rafiei, 2024 ) simplifies SQL generation by fine-tuning schema linking and SQL generation separately, but lacks detailed analysis. CHESS ( Talaei et al., 2024 ) uses a pipeline method to generate SQL in multiple steps, proving the superiority of the pipeline method. MSc-SQL ( Gorti et al., 2024 ) generates candidate SQL by fine-tuning multiple different models, and then uses a selection model to select the final results. Although these methods are comparable to the methods using the closed-source model like GPT-4, they are lower than the methods using the most advanced closed-source models like GPT-4o and Gemini 1.5 pro. At the same time, the SFT-based methods lack a high cost-effectiveness baseline method, which also hinders the development of SFT-based methods. In order to solve the challenges faced by the open source model SFT method, we propose a new method BASE-SQL based on the pipeline method. BASE-SQL mainly consists of four components: Schema Linking, Candidate SQL Generate, SQL Revision and SQL Merge Revision. In the Schema Linking component, we only perform table-linking, not column-linking. Table-linking is obtained by fine-tuning an open source model. Candidate SQL Generate is also obtained by fine-tuning an open source model. SQL Revision uses another open source model combined with Full Schema to perform SQL error correction. Finally, SQL Merge Revision is used to generate the final SQL through two rounds of merge correction. Through detailed experiments, we analyzed the effects of different components under different parameter configurations. Finally, our method BASE-SQL only uses two 32B open source models Qwen2.5-Coder-32B-Instruct ( Hui et al., 2024 ) and Qwen2.5-32B-Instruct ( Yang et al., 2024a ) , and achieved an accuracy of 67.47% on the BIRD ( Li et al., 2024b ) development set and 88.9% on the Spider test set. BASE-SQL is much more efficient than other state-of-the-art methods. On average, it only needs to call the large model 5 times for one generation. In addition, the cost of reproducing BASE-SQL is also very low. There is no need to use additional data for incremental pre-training. In general, BASE-SQL is a cost-effective Text-to-SQL baseline method. In summary, our main contributions are as follows: 1. We analyzed the advantages and disadvantages of the current SFT-based Text-to-SQL methods and proposed a new pipeline-based method BASE-SQL. 2. Through a large number of comparative experiments, we analyzed the SQL generation effects of different components of our method under different parameter configurations, providing a certain reference for subsequent research. 3. We only use two 32B open source models, and on the BIRD and Spider datasets, we outperform all other methods using open source models, and even surpass several methods using GPT-4o closed source models. At the same time, it has high inference efficiency and low reproduction cost, indicating that BASE-SQL is a cost-effective baseline method. Our source code will be published at https://github.com/CycloneBoy/base_sql . 2 Related Work Early Text-to-SQL methods are mainly based on the Seq2Seq method. The natural language and database schema are semantically encoded through the encoder, and then decoded through the decoder to generate the corresponding SQL statement ( Zhong et al., 2017 ) . Since the SQL language has certain grammatical rules, it is easy to make errors when generating SQL models directly through the decoder.

Read the complete text