SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translation

Muhammad Adeel Ijaz · 2026

Read SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translation by Muhammad Adeel Ijaz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16511.txt.

Introduction

Ijaz presents SQL Query Engine, an open-source service that turns plain questions into PostgreSQL queries. Stage one inspects the schema, generates SQL, and parses output from any format without structured-output APIs. Stage two runs the query and, on failure or empty results, enters a self-healing loop where the model diagnoses errors from SQLSTATE codes and diagnostics. Early acceptance prevents regressions, with best-result tracking. Schema context is cached per session, progress streams live, and an OpenAI-compatible endpoint lets tools connect, with all connections read-only.

Catalog record

CREATORMuhammad Adeel Ijaz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50713
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16511

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translation Muhammad Adeel Ijaz Abstract We present SQL Query Engine , an open-source, self-hosted service that translates natural language questions into validated PostgreSQL queries through a two-stage pipeline driven by large language models (LLMs). The first stage performs automatic schema introspection and LLM-guided SQL generation; a multi-strategy response parser extracts SQL from any LLM output format (JSON, code blocks, or raw text) without requiring structured output APIs or function calling. The second stage executes the generated query against a live PostgreSQL instance and, upon failure or empty results, enters an iterative self-healing loop in which the LLM diagnoses the error, using full SQLSTATE codes and PostgreSQL diagnostic messages, and produces a corrected query. Two mechanisms prevent regressions: early-accept returns successful queries immediately without LLM re-evaluation, and best-result tracking preserves the best partial result across retries. The system caches schema context per session in Redis, streams real-time progress events via Redis Pub/Sub and Server-Sent Events (SSE), and exposes both a native REST API and a fully OpenAI-compatible /v1/chat/completions endpoint, so existing tools such as Open WebUI and the OpenAI Python SDK work without modification. All database connections are enforced as read-only at the driver level. We evaluate the pipeline on two fronts: (1) a synthetic ablation study across five LLM backends and 75 gold-standard questions spanning three purpose-built PostgreSQL databases, where the self-healing loop yields up to +9.3 percentage-point accuracy gains with zero regressions on the best model (Llama 4 Scout 17B, 57.3%); and (2) the BIRD benchmark, a large-scale real-world evaluation on 437 questions across 11 databases migrated from SQLite to PostgreSQL, where the full pipeline reaches 49.0% execution accuracy (GPT-OSS-120B) with a +4.6 percentage-point self-healing improvement. Source code is publicly available at https://github.com/codeadeel/sqlqueryengine . Keywords text-to-SQL \cdot large language models \cdot self-healing \cdot PostgreSQL \cdot natural language interfaces \cdot execution-grounded repair \cdot BIRD benchmark 1 Introduction Translating natural language into executable SQL queries, commonly referred to as the text-to-SQL problem, has long been a research objective in the database and natural language processing communities [ 29 , 28 ] . The emergence of large language models (LLMs) such as GPT-4 [ 16 ] and open-weight alternatives like LLaMA [ 24 ] has substantially advanced the state of the art, with recent methods achieving execution accuracies above 85% on the Spider benchmark [ 18 , 9 ] . Yet a considerable gap remains between benchmark performance and reliable deployment in production settings, where schemas are large, dynamic, and undocumented. A core limitation of single-pass text-to-SQL approaches is their inability to recover from errors. When an LLM generates an incorrect query, most systems either return the raw database error to the user or silently give up [ 23 ] . In practice, however, many of these failures are recoverable : the query may contain a misspelled column name, an incorrect type cast, or an overly restrictive filter that a second LLM call could fix if provided with the right diagnostic information. This observation motivates the design of SQL Query Engine , an open-source, self-hosted pipeline that pairs LLM-driven SQL generation with an execution-grounded self-healing loop. Figure 1 summarizes the key result: across two independent evaluation suites and five LLM backends, the self-healing loop consistently recovers failed queries and raises execution accuracy by up to +9.3 percentage points on synthetic benchmarks and +4.6 percentage points on BIRD [ 13 ] , a large-scale real-world benchmark. Config C (generation only) Config A (full self-healing) Avg. latency (s) Synthetic Benchmark: Accuracy vs. Latency (75 questions) Accuracy (%) 0 10 20 30 40 50 60 Latency (s) 0 2 4 6 8 10 45.3 53.3 GPT-OSS 20B 53.3 54.7 Llama 3.3-70B 50.7 48.0 GPT-OSS 120B 48.0 57.3 Llama 4 Scout 48.0 46.7 Qwen3 32B Figure 1: Synthetic benchmark: execution accuracy (bars, left axis) and average query latency (lines, right axis) before and after self-healing. Scout achieves the highest accuracy (57.3%, +9.3pp) with moderate latency increase (1.67s \to 4.19s). The latency cost of self-healing scales with the number of repair iterations needed. Beyond the raw accuracy numbers, three production-oriented challenges motivate the design. First, real-world database schemas are often large and evolve over time; the system must introspect the schema dynamically rather than relying on a static, curated representation. Second, LLM-generated queries frequently fail on the first attempt due to schema mismatches, incorrect type casts, or overly restrictive filters, yet most existing systems either surface the error to the user or silently give up. Third, enterprise environments demand safety guarantees (e.g., that the system cannot modify data) and observability into the query repair process. SQL Query Engine addresses these concerns through three principal design decisions: 1. A two-stage pipeline that separates SQL generation from SQL evaluation and repair, with each stage implemented as a composable module exposable via independent API endpoints. 2. A self-healing loop in the evaluation stage that captures full PostgreSQL error diagnostics (SQLSTATE codes, diagnostic messages, hints, and Python tracebacks) and feeds them back to the LLM for iterative correction, guarded by early-accept and best-result tracking to prevent regressions. 3. A session-aware caching layer backed by Redis that stores per-user schema descriptions and conversation history, eliminating redundant database introspection and enabling multi-turn interactions. The system is LLM-agnostic: it communicates with any OpenAI-compatible endpoint (Ollama, vLLM, OpenAI, LiteLLM, and others), so switching between local open-weight models and hosted APIs requires no code changes. It also exposes an OpenAI-compatible API surface itself, so clients such as Open WebUI [ 15 ] or the OpenAI Python SDK can connect to it transparently. The remainder of this report is organized as follows. Section 2 surveys related work. Section 3 details the system architecture. Section 4 describes the self-healing algorithm. Section 5 covers the real-time streaming protocol. Section 6 describes the dual API surface. Section 7 discusses safety mechanisms. Section 8 presents empirical evaluations on both synthetic and BIRD benchmarks across five LLM backends. Section 9 reflects on design trade-offs. Section 11 concludes. 2 Related Work Text-to-SQL with LLMs. The text-to-SQL field has evolved from sequence-to-sequence models [ 29 ] and relation-aware encoders [ 25 ] toward LLM-based pipelines that use in-context learning and chain-of-thought prompting [ 2 , 27 ] . DIN-SQL [ 18 ] decomposes the task into schema linking, classification, SQL generation, and self-correction sub-tasks, achieving 85.3% execution accuracy on Spider; by breaking the problem into smaller, more tractable sub-problems, DIN-SQL shows that task decomposition can substantially reduce the cognitive load on the LLM at each step. DAIL-SQL [ 9 ] investigates prompt engineering strategies (question representation, example selection, and example organization) and reaches 86.6% through a combination of supervised fine-tuning and in-context learning, establishing that the choice of prompt format matters as much as the choice of model. C3 [ 7 ] demonstrates that a carefully designed zero-shot prompt with ChatGPT can reach 82.3% without any training examples, suggesting that strong base models may not need in-context demonstrations for moderately complex queries. CHASE-SQL [ 17 ] takes a different approach by employing multi-path reasoning: it generates candidate SQL queries through several strategies (divide-and-conquer decomposition, chain-of-thought with execution plans, and instance-aware synthetic examples) and then selects the best candidate via a preference-optimized ranking

Read the complete text