Data Flows in You: Benchmarking and Improving Static Data-flow Analysis on Binary Executables
Nicolaas Weideman 1 , Sima Arasteh 2 , Mukund Raghothaman 2 , Jelena Mirkovic 3 , Christophe Hauser 4 · 2025
Read Data Flows in You: Benchmarking and Improving Static Data-flow Analysis on Binary Executables by Nicolaas Weideman 1 , Sima Arasteh 2 , Mukund Raghothaman 2 , Jelena Mirkovic 3 , Christophe Hauser 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00313.txt.
Introduction
Nicolaas Weideman and co-authors wrote a paper from 2025 on tools that trace how data flows inside computer programs. Such tools help find weak spots that hackers could use. They work on the raw code files that users run, where the flow of data is hard to track. No one had a good gauge of how exact the best tools are.
The team built a huge test set of small programs with known data flows, plus flows drawn from six real programs. They ran three well-known tools on the set and found that all three were far from exact. They then added three fixes to the way the tools model a program. With the fixes, the tools found nearly all true flows and raised the share of right flags a great deal. The fixes also helped find real weak spots.
Catalog record
| CREATOR | Nicolaas Weideman 1 , Sima Arasteh 2 , Mukund Raghothaman 2 , Jelena Mirkovic 3 , Christophe Hauser 4 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00313
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Data Flows in You: Benchmarking and Improving Static Data-flow Analysis on Binary Executables Nicolaas Weideman 1 , Sima Arasteh 2 , Mukund Raghothaman 2 , Jelena Mirkovic 3 , Christophe Hauser 4 Abstract Data-flow analysis is a critical component of security research. Theoretically, accurate data-flow analysis in binary executables is an undecidable problem, due to complexities of binary code. Practically, many binary analysis engines offer some data-flow analysis capability, but we lack understanding of the accuracy of these analyses, and their limitations. We address this problem by introducing a labeled benchmark data set, including 215,072 215,072 microbenchmark test cases, mapping to 277,072 277,072 binary executables, created specifically to evaluate data-flow analysis implementations. Additionally, we augment our benchmark set with dynamically-discovered data flows from 6 6 real-world executables. Using our benchmark data set, we evaluate three state of the art data-flow analysis implementations, in angr, Ghidra and Miasm and discuss their very low accuracy and reasons behind it. We further propose three model extensions to static data-flow analysis that significantly improve accuracy, achieving almost perfect recall (0.99) and increasing precision from 0.13 to 0.32. Finally, we show that leveraging these model extensions in a vulnerability-discovery context leads to a tangible improvement in vulnerable instruction identification. I Introduction With the ever-increasing reliance of the modern world on software systems, the cat-and-mouse game of finding and mitigating software vulnerabilities before they are exploited has reached critical levels. This is exacerbated by the growing complexity of software, which increases the burden of ensuring its security. To assuage this burden, program analysis for system security has gained more traction. Data-flow analysis is a critical component in program analysis for security, as understanding how information flows between the instructions of a program is often necessary to evaluate the program’s security. Unfortunately, static data-flow analysis is undecidable in the general case. Moreover, when evaluating the security of a program, analyzing the source code is often considered to be insufficient due to the What You See is not What You Execute phenomenon [ 1 ] . This creates the necessity for accurate data-flow analysis designed to operate on the binary instructions of programs. Fortunately, while data-flow analysis may be undecidable in a general case, there are many practical scenarios in which it is possible to perform data-flow analysis on small code segments (e.g., within a function), by implementing certain approximations in binary analysis engines [ 2 , 3 , 4 ] . These approximations manifest as assumptions made in the model and heuristics incorporated into the implementation of the binary analysis engine. The approximations may at times sacrifice correctness in order to achieve guaranteed termination (and indeed scalability). Incorrect data-flow analysis manifests as false positives (non-existing data flows identified) and false negatives (existing data flows missed). Unfortunately, there is currently no systematic approach to evaluate the size and scope of inaccuracies in binary engines’ implementations of data flow analysis, and to identify opportunities for improvements. This paper aims to benchmark data-flow approaches in binary analysis engines, to produce comprehensive, quantitative and insightful findings about their strengths and limitations. The benchmark dataset should not only measure accuracy of a data-flow analysis approach, but it should also identify root causes for inaccuracies (false positives and false negatives), which can be tackled by future researchers. We demonstrate how the insights from our evaluation led us to propose three improvements to angr’s data-flow analysis model, which improved recall from 40% to 99% and precision from 13% to 32%. Improving data-flow analysis accuracy is an essential step towards improving binary program analysis for system security. This paper demonstrates such an improvement in the context of vulnerability discovery, showing that our improvements to angr lead to higher identification of vulnerable data flows in three case studies. In summary, this paper makes the following contributions. • We introduce alias classes , a novel concept for categorizing data flows. • We define and implement an open source framework for generating microbenchmark test cases, labeled with ground truth, that can be used to evaluate static data flow analysis models with respect to our alias classes. • We define and implement a framework for extracting data flows from real-world programs using dynamic analysis, to generate ground-truth information about existing data flows. • With the provided frameworks, we generate a data-flow evaluation benchmark data set, consisting of 215,072 microbenchmark test cases mapping to 277,072 unique binary executables, as well as data flows from 6 real-world executables. • We show the twofold utility of this data set: 1. We evaluate three state of the art static data-flow analysis implementations in angr [ 2 ] , Ghidra [ 3 ] and Miasm [ 4 ] . Our evaluation yields insights about how accurate these analysis engines are with respect to different categories of data flows, and enables us to identify areas for improvement. To the best of our knowledge, this is the first evaluation of its kind. 2. We propose three novel data-flow model extensions, implement them on top of angr, and evaluate them on real-world executables. These extensions achieve nearly perfect recall, while simultaneously improving precision of data-flow analysis on our benchmarks. 3. We show that leveraging our model extensions in a vulnerability-discovery context leads to an improved recovery of vulnerability-related instructions in three case studies. II Binary-level data-flow analysis To perform data-flow analysis on a program is to reason about the flow of information between its instructions. Since data flow between two instructions requires executing these instructions in order, data-flow analysis is often built on top of control-flow analysis. Consequently, data-flow analysis inherits the complexities of control-flow analysis, such as context sensitivity – the notion that the next instruction in an execution path may be determined by the preceding instructions. For example, the instruction following a return instruction, is determined by the preceding, matching call instruction. This complexity is part of a larger problem caused by indirect branches, in which case the control-flow depends on the data flow. Due to the intermixed nature of control-flow and data-flow analysis, in the general case, both problems are undecidable [ 5 ] . Moreover, performing these analyses on binary code, as opposed to source code, adds an extra layer of complexity due to the loss of high-level semantic information, such as control-flow structures, data types and data structures. However, binary code is the most faithful program representation when it comes to reasoning about a range of security properties, and it is therefore our target. One of the root causes of the undecidability of data-flow analysis lies in the aliasing problem , i.e., determining if two instructions access the same data ( i.e. , if two different pointers point to the same location). In binary program analysis, this means determining if a memory write instruction and a subsequent memory read instruction access memory at the same address. A data flow exists between these instructions if a control flow exists and if they must (or may) access the same data, such that the data written by the first instruction is read by the second instruction, and has not been changed in the meantime. Stated differently, a data flow exists between two instructions if they form a link in a def-use chain [ 6 ]