QLCoder: A Query Synthesizer for Static Analysis of Security Vulnerabilities

Claire Wang, Ziyang Li, Saikat Dutta, Mayur Naik · 2025

Read QLCoder: A Query Synthesizer for Static Analysis of Security Vulnerabilities by Claire Wang, Ziyang Li, Saikat Dutta, Mayur Naik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08462.txt.

Introduction

Claire Wang, Ziyang Li, Saikat Dutta, and Mayur Naik present QLCoder, an agent system that writes CodeQL security queries automatically from vulnerability records. Static analysis can hunt vulnerable code patterns, but writing good queries demands rare combined expertise in security and program analysis. QLCoder places a language model inside a synthesis loop with execution feedback, guided by a custom interface to a language server for syntax help and a retrieval database for examples and documentation.

Evaluation covers one hundred seventy six known vulnerabilities across one hundred eleven Java projects. For over half of them, the system produces a correct query that flags the vulnerable version and not the patched one. A plain agent baseline manages only about a tenth, showing the value of the structured guidance.

Catalog record

CREATORClaire Wang, Ziyang Li, Saikat Dutta, Mayur Naik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08462

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

QLCoder: A Query Synthesizer for Static Analysis of Security Vulnerabilities Claire Wang, Ziyang Li, Saikat Dutta, Mayur Naik Abstract Static analysis tools provide a powerful means to detect security vulnerabilities by specifying queries that encode vulnerable code patterns. However, writing such queries is challenging and requires diverse expertise in security and program analysis. To address this challenge, we present QLCoder – an agentic framework that automatically synthesizes queries in CodeQL, a powerful static analysis engine, directly from a given CVE metadata. QLCoder embeds an LLM in a synthesis loop with execution feedback, while constraining its reasoning using a custom MCP interface that allows structured interaction with a Language Server Protocol (for syntax guidance) and a RAG database (for semantic retrieval of queries and documentation). This approach allows QLCoder to generate syntactically and semantically valid security queries. We evaluate QLCoder on 176 existing CVEs across 111 Java projects. Building upon the Claude Code agent framework, QLCoder synthesizes correct queries that detect the CVE in the vulnerable but not in the patched versions for 53.4% of CVEs. In comparison, using only Claude Code synthesizes 10% correct queries. Our generated queries achieve an F1 score of 0.7. In comparison, the general query suites in IRIS (a recent LLM-assisted static analyzer) and CodeQL only achieve F1 scores of 0.048 and 0.073, highlighting the benefit of QLCoder’s specialized synthesized queries. 1 Introduction Security vulnerabilities continue to grow at an unprecedented rate, with over 40,000 Common Vulnerabilities and Exposures (CVEs) reported in 2024 alone ( 4 ) . Static analysis, a technique to analyze programs without executing them, is a common way of detecting vulnerabilities. Static analysis tools such as CodeQL ( 6 ) , Semgrep ( 18 ) , and Infer ( 14 ) are widely used in industry. They provide domain-specific languages that allow specifying vulnerability patterns as queries. Such queries can be executed over structured representations of code, such as abstract syntax trees, to detect potential security vulnerabilities. Despite their widespread use, existing query suites of static analysis tools are severely limited in coverage of vulnerabilities and precision. Extending them is difficult even for experts, as it requires knowledge of unfamiliar query languages, program analysis concepts, and security expertise. Incorrect queries can produce false alarms or miss bugs, limiting the effectiveness of static analysis. Correct queries can enable reliable detection of real vulnerabilities, supporting diverse use-cases such as regression testing, variant analysis, and patch validation, among others (Figure 1 ). Meanwhile, CVE databases ( 15 ; 17 ; 10 ) provide rich information about security vulnerabilities, including natural language descriptions of vulnerability patterns and records of buggy and patched versions of the affected software repositories. This resource remains largely untapped in the automated construction of static analysis queries. Recent advances in LLMs, particularly in code understanding and generation, open up the possibility of leveraging this information to automatically synthesize queries from CVE descriptions, thereby bridging the gap between vulnerability reports and practical detection tools. Synthesizing such queries poses significant challenges. The syntax of static analysis query languages is low-resource, richly expressive, and evolves continually. A typical query, such as the one in Figure 2 (b) specifying a global dataflow pattern leaves ample room for errors in describing predicates for sources, sinks, sanitizers, and taint propagation steps. Even if the generated syntax is correct, success is measured by whether the query can identify at least one execution path traversing the bug location in the vulnerable version while producing no matches in the patched version. Achieving this requires understanding the CVE context at the level of abstract syntax trees, such as code differences that introduce a sanitizer to prevent a flow from a source to a sink. Complicating matters further, reasoning about the code changes alone is often insufficient: sources, sinks, and taint propagation steps may reside in parts of the codebase far from the modified functions or files, and the vulnerability itself may involve non-trivial dataflow chains across these components. Thus, a correct query must not only integrate information from multiple locations across the program but also capture the intricate propagation patterns to accurately characterize the vulnerability. Figure 1: A CodeQL query capturing a vulnerability pattern is synthesized by QLCoder from an existing CVE and subsequently reused for regression testing, variant analysis, or patch validation. In this paper, we present QLCoder – an agentic framework that synthesizes queries in CodeQL, a powerful static analysis engine, directly from a given CVE metadata. We select CodeQL because it has the richest query language, which allows capturing complex inter-procedural vulnerability patterns. QLCoder addresses the above challenges by embedding an LLM in a structured synthesis loop that incorporates execution feedback to verify query correctness and allows interactive reasoning using a custom MCP (Model Context Protocol) interface. The MCP interface constrains the model’s reasoning using a Language Server Protocol (for syntax guidance) and a vector database of CodeQL queries and documentation (for semantic guidance). By combining these capabilities, QLCoder avoids common pitfalls of naive LLM-based approaches, such as producing ill-formed queries, hallucinating deprecated constructs, or missing subtle vulnerability patterns, and instead produces queries that are both syntactically correct and semantically precise. We evaluate QLCoder on CWE-Bench-Java ( 13 ) , which comprises 176 CVEs across 111 Java projects. These CVEs span 42 different Common Weakness Enumeration (CWE) categories and the projects range in size from 0.01 to 1.5 MLOC. To account for model training cut-offs, we include 65 CVEs reported during 2025 and target a recent CodeQL version 2.22.2 (July 2025). Using the Claude Code agent framework, QLCoder achieves query compilation and success rates of 100% and 53.4%, compared to 19% and 0% for our best agentic baseline, Gemini CLI. Further, our generated queries have an F1 score of 0.7 for detecting true positive vulnerabilities, compared to 0.048 for IRIS ( 13 ) , a recent LLM-assisted static analyzer, and 0.073 for CodeQL. We summarize our main contributions: • Agentic Framework for CVE-to-Query Synthesis. We present QLCoder, an agentic framework that translates CVE descriptions into executable CodeQL queries, bridging the gap between vulnerability reports and static analysis. QLCoder introduces a novel integration of execution-guided synthesis, semantic retrieval, and structured reasoning for vulnerability query generation. • Evaluation on Real-World Repositories and CVEs. We evaluate QLCoder on 176 CVEs in Java projects, covering 42 vulnerability types (CWEs) from CWE-Bench-Java. Each project involves complex inter-procedural vulnerabilities spanning multiple files. We show how QLCoder can successfully identify sources, sinks, sanitizers, and taint propagation steps, and refine queries to ensure they raise alarms on vulnerable versions while remaining silent on patched versions. • Comparison with Baselines. We compare QLCoder against state-of-the-art agent frameworks and show that QLCoder achieves substantially higher compilation, success, and F1 scores. We also compare QLCoder’s synthesized queries with state-of-the-art static analysis frameworks and show that our queries are more precise and have higher recall. 2 Illustrative Example We illustrate the challenges of vulnerability query synthesis using CVE-2025-27136, an XML External Entity Injection (XXE) bug found in the repository Robothy/local-s3 . Figure 2 depicts the vulnerability snippets, the patch, and the synthesized CodeQL query generated by QLCoder. (a) The vulnerable dataflow snippets and the patch, which adds configuration to XMLInputFactory . (b) Snippets of the synthesized vulnerability query by QLCoder capturing

Read the complete text