GradeSQL: Outcome Reward Models for Ranking SQL Queries from Large Language Models
Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia · 2025
Read GradeSQL: Outcome Reward Models for Ranking SQL Queries from Large Language Models by Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01308.txt.
Introduction
Turning a plain question into a database query, called text-to-SQL, has improved with large language models, yet complex queries still trip them up. Mattia Tritto, with six co-authors, studies how to pick the best answer when a model makes several tries at test time. Common methods favor the query that runs without error, or the answer given most often.
The team argues outcome reward models, scorers trained to judge whether an output is truly right, can serve as a better test-time guide. Their work, GradeSQL, explores that largely untested use for SQL generation. By scoring the meaning of each candidate query rather than its surface form, the method aims to lift accuracy on hard questions. Better selection could widen safe access to databases for non-expert users.
Catalog record
| CREATOR | Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GradeSQL: Outcome Reward Models for Ranking SQL Queries from Large Language Models Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia Abstract . Text-to-SQL, the task of translating natural language questions into SQL queries, has significantly advanced with the introduction of Large Language Models (LLMs), broadening database accessibility for a wide range of users. Despite substantial progress in generating valid SQL, current LLMs still struggle with complex queries that require precise alignment between user intent and the database schema. To mitigate this, test-time strategies such as Best-of- N N (BoN) and Majority Voting (Maj) are often employed, based on the assumption that LLMs can generate correct answers but may require multiple attempts. However, these methods rely on surface-level heuristics, selecting either the syntactically correct query through execution-based BoN (ex-BoN) or the most frequently generated query with Maj. Recently, Outcome Reward Models (ORMs), which assign utility scores to generated outputs based on semantic correctness, have emerged as a promising approach for better aligning model predictions with user intent. Nevertheless, their application to Text-to-SQL remains largely underexplored. In this work, we evaluate ORMs as an effective heuristic for BoN, compare them with ex-BoN and Maj, and introduce a framework for training ORMs for the Text-to-SQL task. We evaluate our ORMs on the BIRD and SPIDER benchmarks, finetuning various open-source LLMs, including the Qwen2, Granite3, and Llama3 model families. Our results show that ORMs outperform ex-BoN and Maj, achieving execution accuracy gains of +4.33% (BIRD) and +2.10% (Spider) over ex-BoN, and +2.91% (BIRD) and +0.93% (Spider) over Maj. We further demonstrate that finetuning models already aligned with SQL generation, such as OmniSQL, yields superior ORM performance. Additionally, we observe that ORMs achieve competitive results on simple queries and benefit more from an increased number of candidates compared to ex-BoN and Maj. All code, datasets, and trained models are publicly released to support reproducibility and encourage future research in this area. † † authors: . PVLDB Reference Format: PVLDB, 19(1): XXX-XXX, 2025. doi:XX.XX/XXX.XX † † This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing [email protected] . Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 19, No. 1 ISSN 2150-8097. doi:XX.XX/XXX.XX PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/sisinflab/GradeSQL . 1. Introduction Text-to-SQL, the task of translating Natural Language (NL) questions into executable SQL queries, facilitates the use of data management systems ( Nascimento et al. 2024 ) and represents a step toward democratizing database access ( Kim et al. 2020 ) , enabling both expert and non-expert users to interact with complex database systems through NL ( Androutsopoulos et al. 1995 ; Li et al. 2024a ; Liu et al. 2025 ) . Early approaches were predominantly rule-based, relying on manually crafted rules and heuristics to map NL questions to SQL queries ( Li and Jagadish 2014a ; Mahmud et al. 2015 ; Li and Jagadish 2014b ) . While often capable of generating syntactically correct queries, these systems struggled with challenges such as nested clauses, co-references, and complex aggregations ( Hong et al. 2024b ) . The field progressed with the adoption of neural network-based techniques ( Xiao et al. 2016 ; Bogin et al. 2019 ) , which improved intent recognition and semantic parsing. This was followed by the introduction of Pre-trained Language Models (PLMs) such as BERT ( Devlin et al. 2019 ) and T5 ( Raffel et al. 2020 ) , which achieved competitive results across various benchmarks ( Li et al. 2023d ; Li et al. 2023b ) . Nevertheless, PLM-based systems still face difficulties with schema generalization and complex queries ( Liu et al. 2025 ) . Recently, the advent of Large Language Models (LLMs) has introduced a new paradigm in Text-to-SQL. LLMs demonstrate strong emergent capabilities that surpass PLMs across diverse NLP tasks ( Kumar 2024 ) , and they currently dominate research in this area ( Pourreza and Rafiei 2023 ; Gao et al. 2024 ; Li et al. 2024c ; Zhang et al. 2024 ) . Nevertheless, LLMs still face challenges in generating correct SQL for complex queries involving multi-table joins, nested subqueries, and sophisticated aggregations ( Ma et al. 2024 ; Kanburoglu and Tek 2024 ) . To address these limitations, researchers are exploring techniques such as prompt engineering ( Gao et al. 2024 ) , multi-agent frameworks ( Wang et al. 2025 ; Askari et al. 2025 ) , and fine-tuning ( Li et al. 2024c ; Qin et al. 2025 ) . While these approaches have driven notable advances, achieving further improvements remains challenging due to the inherent complexity of the task. Inspired by recent advances in mathematical reasoning with LLMs ( Zhang et al. 2025 ) , the research community is increasingly investigating test-time inference strategies, which allocate additional computational resources at inference to enhance output quality ( Li et al. 2025b ; He et al. 2025 ; Brown et al. 2024 ) . These strategies typically generate multiple candidate outputs and apply heuristics to select the best one, leveraging the ability of LLMs to produce correct answers when afforded more inference time rather than larger model size or additional training ( Xie et al. 2025 ) . Common approaches include Best-of- N (BoN) ( Cobbe et al. 2021 ) , which samples N outputs and selects the highest-scoring candidate, and Majority Voting (Maj) ( Sheng and Xu 2025 ) , which chooses the most frequently generated output. However, these methods often depend on shallow heuristics such as frequency or execution success, without explicitly assessing semantic correctness. To address this gap, recent work has looked beyond surface-level heuristics toward methods that explicitly evaluate the semantic quality of candidate outputs. In particular, the Reinforcement Learning (RL) community has proposed Outcome Reward Models (ORMs), which assign fine-grained semantic correctness scores to candidate outputs ( Zhang et al. 2025 ) . When used as reward functions in RL loops, ORMs improve the alignment between model predictions and user intent ( Havrilla et al. 2024 ) , highlighting their potential as principled verifiers of model output. This naturally raises the question: can ORMs also enhance test-time inference strategies? In particular, integrating ORMs into BoN offers a way to move beyond frequency- or execution-based heuristics, enabling candidate selection grounded in semantic correctness. Although BoN and Maj have already been applied to Text-to-SQL ( Li et al. 2025b ) , the use of ORMs in test-time inference remains largely unexplored, hindered by the scarcity of reward-labeled data and the absence of clear methodological guidelines. Moreover, no systematic evaluation has yet compared ORM-guided selection with traditional test-time compute strategies, leaving open the potential of ORMs as a more robust heuristic for BoN. To address these limitations, this study presents a comprehensive evaluation of BoN guided by ORMs, comparing it against execution-based BoN and Maj. We further introduce a scalable data synthesis framework tailored to the training requirements of ORMs in the Text-to-SQL setting. This framework automates data