Minimizing False Positives in Static Bug Detection via LLM-Enhanced Path Feasibility Analysis

Xueying Du, Chong Wang, Wentai Deng, Zuoyu Ou Xin Peng Lingming Zhang Yiling Lou · 2025

Read Minimizing False Positives in Static Bug Detection via LLM-Enhanced Path Feasibility Analysis by Xueying Du, Chong Wang, Wentai Deng, Zuoyu Ou Xin Peng Lingming Zhang Yiling Lou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-10322.txt.

Introduction

Static bug hunters cry wolf so often that real bugs hide in the noise. The authors, a nine-name team writing in 2025, send a language model in to check each flagged path before a human must. The catalog's author string runs the names together and drops two; the head byline lists all nine. Their frame works in rounds: an agent reasons over the locks a path must pass, guided by a plan and by the code around the flag.

Across their tests the frame threw out between seven and nineteen of every twenty false flags. That beat all rivals by wide marks. Of forty-five true bugs in the pile, it missed just three. The team pitch the round-by-round check as the fix for big code bases, where past model passes ran out of steam.

Catalog record

CREATORXueying Du, Chong Wang, Wentai Deng, Zuoyu Ou Xin Peng Lingming Zhang Yiling Lou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-10322

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Minimizing False Positives in Static Bug Detection via LLM-Enhanced Path Feasibility Analysis Xueying Du, Chong Wang, Wentai Deng, Zuoyu Ou Xin Peng Lingming Zhang Yiling Lou Abstract Static bug analyzers play a crucial role in ensuring software quality. However, existing analyzers for bug detection in large codebases often suffer from high false positive rates. This is primarily due to the limited capabilities of analyzers in path feasibility validation with multiple conditional branches and complex data dependencies. While current LLM-based approaches attempt to address this issue, their effectiveness remains limited due to insufficient constraint cascade analysis and scalability challenges in large projects. To address this challenge, we propose an iterative path feasibility analysis framework LLM4PFA . By leveraging LLM agent based targeted constraint reasoning, and key context-aware analysis driven by agent planning, LLM4PFA effectively enhances complex inter-procedural path feasibility analysis for minimizing false positives in static bug detection. Evaluation results show that LLM4PFA precisely filters out 72% to 96% false positives reported during static bug detection, significantly outperforming all the baselines by 41.1% - 105.7% improvements; meanwhile LLM4PFA only misses 3 real bugs of 45 true positives. 1 Introduction Static bug analyzers play a crucial role in ensuring software security, reliability, and maintainability. They help identify potential bugs early in the development process by scanning the codebase without extensive program execution Cai et al. 2023 ; Cai et al. 2022 ; Cai et al. 2021 ; Shi et al. 2020 ; Shi et al. 2018 ; Sui et al. 2012 ; Vassallo et al. 2020 ; Yan et al. 2018 . Popular analyzers like GitHub CodeQL cod 2025 , Facebook Infer inf 2025 , and CppCheck cpp 2024 are widely used for industrial bug detection. Their analysis of many bug types (e.g., null pointer dereference and use-after-free) typically follows a standardized approach: identifying sources (i.e., potential origins of issues) and sinks (i.e., locations where issues manifest) based on predefined rules, then checking whether a feasible execution path exists from sources to sinks. If such a path is found, the tools flag a potential bug. However, in complex software systems, sources and sinks often reside in different functions. In cases where function call chains between a source and a sink are lengthy and involve multiple conditional branches and complex data dependencies, traditional static analyzers struggle to comprehensively validate path feasibility between sources and sinks, resulting in high false positive rates. For instance, our manual annotation of sampled bug reports reveals that popular tools produce an extremely high rate (e.g., over 90%) of false positives for use-after-free bugs across large-scale software such as Linux Kernel. This significantly burdens developers due to the large efforts in manually reviewing the false alarms reported by these tools. Existing Techniques. As Large Language Models (LLMs) are increasingly applied to code-related tasks Du et al. 2024a ; Liu et al. 2024 ; Nam et al. 2024 ; Du et al. 2024b , researchers have leveraged their advanced program understanding capabilities to reduce false positives generated by static analyzers. However, existing LLM-based approaches have limited effectiveness in handling path feasibility analysis within large, dependency-rich codebases. Specifically, LLM4SA Wen et al. 2024 feeds entire source-to-sink call chains along with bug reports into LLMs at once to identify false positives, without conducting fine-grained constraint analysis and relying entirely on the LLM’s ability to assess complex path feasibility. LLMDFA Wang et al. 2024a performs data flow analysis for each function along all potential execution paths between sources and sinks, which results in significant time and computational cost when applied to large-scale projects with complex call dependencies. While symbolic execution can perform path feasibility analysis, it suffers from path explosion when traversing all possible paths, particularly when dealing with multiple conditional branches, loops, nested function invocations, or compound data types Baldoni et al. 2018 . Therefore, it cannot scale to large and complex software systems in practice. Our approach. To address these limitations, we propose LLM4PFA , an LLM-powered path feasibility analysis framework designed to minimize false positives in static bug detection for complex software projects. Specifically, Given a function call trace and a target variable with its source and sink, LLM4PFA iteratively performs two-phase analysis for each function: (1) Feasibility Constraints Extraction first identifies critical conditional expressions impacting sink reachability; then employs LLM agents to conduct symbolic range reasoning for variables and function calls within each expression. (2) Constraints Solving employs constraint solver to iteratively solve the SMT query scripts generated by LLM. Overall, LLM4PFA offers the following advantages compared to traditional symbolic execution and existing LLM-based methods: • Precise complex constraint cascade analysis. Breaking down inter-procedural feasibility analysis into localized symbolic range reasoning, LLM4PFA amplifies LLMs’ program comprehension strength on focused code segments Li et al. 2023b , thereby significantly enhancing reasoning effectiveness. • Efficient complex context analysis. Integrating prioritized feasible path constraints extraction method with agent-guided planning, LLM4PFA effectively mitigates path explosion in sequentially traversing deeply-nested contextual functions. Evaluation. We manually construct a new dataset, SAFP-BENCH-C, for a more realistic evaluation, which takes approximately 100 person-hours. It contains 364 warnings generated by three advanced static analyzers, covering three important bug types (i.e., null pointer dereference, buffer overflow, and use after free) in large-scale software projects, including the Linux Kernel (with over 18 million lines of code), OpenSSL (with over 300K lines of code), and Libav (with over 600K lines of code). Our evaluation shows that LLM4PFA precisely filters out 72% to 96%) false positives reported during static bug detection, significantly outperforming all the baselines by 41.1% - 105.7% improvements; meanwhile LLM4PFA only misses 3 real bugs of 45 true positives. Furthermore, LLM4PFA shows consistent effectiveness across four state-of-the-art LLMs, indicating the generalizability of our approach. Ablation studies further reveal that LLM4PFA outperforms two ablation variants, underscoring the key contribution of LLM agent based self-planning context analysis and iterative constraint solving proposed in our framework. This paper makes the following contributions: • A novel technique that effectively enhances complex inter-procedural feasibility analysis to minimize false positives in bug detection. • A new dataset designed for a more realistic evaluation of false positive reduction, with approximately 100 person-hours dedicated to manual construction. • An extensive evaluation demonstrating the effectiveness of LLM4PFA in filtering out false positives and identifying real bugs. (a) Complex Constraint Cascades (b) Complex Constraint-Related Contextual Analysis Figure 1: Motivating Examples 2 Preliminaries & Motivation This section introduces key preliminaries of static bug detection and presents motivating examples to illustrate the false positive issues related to path feasibility validation. 2.1 Static Bug Detection Many bugs, such as Null Pointer Dereference (NPD) and Use After Free (UAF), follow a common pattern where a target variable feasibly flows from a source (where a potential issue originates) to a sink (where the issue manifests). Source and Sink Identification. The specific definitions of the target variable, source, and sink vary by bug type. For instance, in NPD (Null Pointer Dereference) bugs, the target variable is a pointer (e.g., tun_info in Figure 1(a) ), the source is the code expression that defines or assigns it as NULL (e.g., Line 4 in Figure 1(a) ), and the sink is the code expression that dereferences the pointer (e.g., Line 105 in Figure 1(a) ). For a specific

Read the complete text