SQL-Commenter: Aligning Large Language Models for SQL Comment Generation with Direct Preference Optimization Conference: 34th IEEE/ACM International Conference on Program Comprehension; April 12–13, 2026; Rio de Janeiro, Brazil 34th IEEE/ACM International Conference on Program Comprehension (ICPC ’26), April 12–13, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3794763.3794803 ISBN: 979-8-4007-2482-4/2026/04 CCS: Software and its engineering CCS: Computing methodologies Natural language processing
Lei Yu, Peng Wang, Jingyuan Zhang, Xin Wang, Jia Xu, Li Yang, Changzhi Deng, Jiajia Ma, Fengjun Zhang · 2026
Read SQL-Commenter: Aligning Large Language Models for SQL Comment Generation with Direct Preference Optimization Conference: 34th IEEE/ACM International Conference on Program Comprehension; April 12–13, 2026; Rio de Janeiro, Brazil 34th IEEE/ACM International Conference on Program Comprehension (ICPC ’26), April 12–13, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3794763.3794803 ISBN: 979-8-4007-2482-4/2026/04 CCS: Software and its engineering CCS: Computing methodologies Natural language processing by Lei Yu, Peng Wang, Jingyuan Zhang, Xin Wang, Jia Xu, Li Yang, Changzhi Deng, Jiajia Ma, Fengjun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18606.txt.
Introduction
Yu and co-authors train a language model to write clear comments for complex SQL queries. Poor comments slow code reading, upkeep, and knowledge transfer. They build a dataset of hard queries with expert-checked comments. Continual pre-training on SQL text and fine-tuning come first. Direct preference optimization with human feedback then sharpens fine semantic choices. On Spider and BIRD, the system beats the strongest baseline by clear margins on three text metrics. Human raters also judge its comments best on correctness, completeness, and naturalness.
Catalog record
| CREATOR | Lei Yu, Peng Wang, Jingyuan Zhang, Xin Wang, Jia Xu, Li Yang, Changzhi Deng, Jiajia Ma, Fengjun Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54211 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18606
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SQL-Commenter: Aligning Large Language Models for SQL Comment Generation with Direct Preference Optimization Conference: 34th IEEE/ACM International Conference on Program Comprehension; April 12–13, 2026; Rio de Janeiro, Brazil 34th IEEE/ACM International Conference on Program Comprehension (ICPC ’26), April 12–13, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3794763.3794803 ISBN: 979-8-4007-2482-4/2026/04 CCS: Software and its engineering CCS: Computing methodologies Natural language processing Lei Yu, Peng Wang, Jingyuan Zhang, Xin Wang, Jia Xu, Li Yang, Changzhi Deng, Jiajia Ma, Fengjun Zhang Abstract . SQL query comprehension is a significant challenge in database and data analysis environments due to complex syntax, diverse join types, and deep nesting. Despite its critical role in backend development and data science, many queries, particularly within legacy systems, often lack adequate comments, which severely hinders code readability, maintainability, and knowledge transfer. Existing approaches to automated SQL comment generation face two main challenges: limited training datasets that inadequately represent real-world analytical queries involving multi-table joins, window functions, and complex aggregations, and an insufficient understanding of SQL-specific logical semantics and schema-related context by Large Language Models (LLMs), even after standard training. Our empirical analysis shows that even after continual pre-training and supervised fine-tuning, LLMs struggle to precisely understand complex SQL semantics, leading to inaccurate or incomplete comments. To address these challenges, we propose SQL-Commenter, an advanced comment generation method based on LLaMA-3.1-8B. First, we construct a comprehensive dataset containing longer, more complex SQL queries with expert-verified, detailed comments. Second, we perform continual pre-training using a large-scale SQL corpus to enhance the LLM’s understanding of SQL syntax and semantics. Then, we conduct supervised fine-tuning with our high-quality dataset. Finally, we introduce Direct Preference Optimization (DPO), which leverages human feedback to significantly improve comment quality. SQL-Commenter utilizes a preference-based loss function that encourages the LLM to increase the probability of preferred outputs while decreasing the probability of non-preferred outputs, thereby enhancing both fine-grained semantic learning, such as distinguishing between different join types, and context-dependent quality assessment based on business logic. We evaluate SQL-Commenter on the authoritative Spider and Bird benchmarks, where it significantly outperforms state-of-the-art baselines. On average, across these datasets, our method surpasses the strongest baseline (Qwen3-14B) by 9.29, 4.99, and 13.23 percentage points on BLEU-4, METEOR, and ROUGE-L, respectively. Moreover, human evaluation demonstrates the superior quality of comments generated by SQL-Commenter in terms of correctness, completeness, and naturalness. † † cc-license: by 1. Introduction SQL (Structured Query Language) remains one of the most widely adopted programming languages, utilized by over half (51.52%) of professional developers ( Hong et al. 2025 ) . However, its syntactic complexity, manifested through intricate join operations, deeply nested subqueries, and complex business logic, creates substantial comprehension challenges, even for experienced practitioners ( Quora 2025 ) . Exacerbating this challenge is the finding that only about a third (35.29%) of practitioners report receiving systematic training ( Stack Overflow 2023 ) . This problem is particularly acute in large-scale legacy systems, where SQL queries often lack adequate comments, severely hindering code readability, maintainability, and knowledge transfer. The consequences of misinterpreting SQL queries are significant, potentially leading to operational inefficiencies ( Yan et al. 2017 ) , security vulnerabilities ( Thomas et al. 2009 ) , and data integrity issues ( Muse et al. 2020 ) . The urgency and scale of this comprehension barrier are starkly illustrated on platforms like Stack Overflow; as of October 19, 2025, a search for the "sql" tag reveals over 674,654 related questions, highlighting a persistent and widespread need for assistance ( Stack Overflow 2025 ) . Translating SQL queries into natural language (SQL-to-Text) offers a practical solution to these challenges ( Ma et al. 2021 ) . For technical professionals, it provides detailed explanations to verify logic and mitigate risks, while for non-technical users, it enhances the accessibility and usability of databases. Despite this clear need, and unlike general-purpose programming languages (GPPLs) where automated comment generation is well-studied ( Geng et al. 2024 ; Li et al. 2022 ; Lu et al. 2023 ; Lu et al. 2025 ) , domain-specific languages (DSLs) such as SQL remain largely unexplored in this context, presenting a significant research gap. Recent benchmarks for Large Language Models (LLMs) have explicitly evaluated the SQL-to-Text task, but their framing reveals a fundamental limitation. For instance, a comprehensive study by Zhang et al. ( Zhang et al. 2024c ) assesses various LLMs (Codellama ( Roziere et al. 2023 ) , InternLM ( Team 2023 ) , InternLM2 ( Cai et al. 2024 ) , Llama2 ( Touvron et al. 2023 ) ) on their ability to transform a SQL query back into its "original natural language question" ( Shu et al. 2021 ) . A critical observation is that these models, whether general-purpose (Llama2, InternLM) or code-specialized (Codellama), are used off-the-shelf without any specialized training or fine-tuning for the SQL-to-Text task. Their evaluation, based on metrics like ROUGE ( Lin 2004 ) and BertScore ( Zhang et al. 2019 ) , focuses on how closely the generated text matches the user’s initial query. This "SQL-to-Question" task demonstrates an LLM’s semantic understanding but fundamentally fails to address a developer’s needs. The output is a high-level summary of the query’s goal, not a technical explanation of its execution, omitting crucial details about join strategies, subquery mechanics, or complex filtering logic. On the other hand, while graph-based models (Graph2Seq, GNNs, HeSQLNet) attempted to better capture SQL’s structure by parsing it into an Abstract Syntax Tree (AST) ( Xu et al. 2018a ; Ma et al. 2021 ; Zhang et al. 2025a ) , the ’text’ they generate is similarly confined to high-level questions or descriptions, sharing the same fundamental limitation. Furthermore, these methods introduce their own complexities, requiring a rigid, intermediate graph representation that can be brittle and may fail to capture all semantic nuances, leading to structural ambiguities. For these reasons, our approach bypasses complex graph-encoding and instead enhances the inherent code understanding of a LLM through targeted post-training, enabling it to generate true technical explanations, not just questions. LLMs have garnered attention for their powerful natural language understanding and generation capabilities, demonstrating remarkable performance in various complex tasks, including code comprehension and generation ( Li et al. 2023b ; Liu et al. 2023b ; Vaithilingam et al. 2022 ; Weyssow et al. 2025 ; Lu et al. 2023 ; Cheng et al. 2025 ; Shen et al. 2024 ; Zan et al. 2024 ) , smart contract auditing and generation ( Yu et al. 2024 ; Yu et al. 2025b ; Yu et al. 2023a ; Yu et al. 2025a ; Yuan et al. 2025b ; Yu et al. 2025c ; Yuan et al. 2025a ) . However, they often encounter significant difficulties in handling SQL-specific concepts and schema-related context, a challenge starkly illustrated by the example in Figure 1 . When presented with a complex query to find a card from the artist who created the most promotional items, a general-purpose model like LLaMA-3.1-8B-Instruct completely misunderstands its logic, incorrectly inferring that the goal is to find the card with the "most ruling information." While a specialized model after continual pre-training and supervised fine-tuning (CPT+SFT) shows improvement by correctly identifying the subquery’s purpose, it still produces deficient comments. As shown in