Symetra : Visual Analytics for the Parameter Tuning Process of Symbolic Execution Engines Volume: 45 Issue: 3
D. Hong, M. Kim, S. Cha, and J. Jo Sungkyunkwan University, Republic of Korea · 2026
Read Symetra : Visual Analytics for the Parameter Tuning Process of Symbolic Execution Engines Volume: 45 Issue: 3 by D. Hong, M. Kim, S. Cha, and J. Jo Sungkyunkwan University, Republic of Korea (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05349.txt.
Introduction
Hong, Kim, Cha, and Jo tackle a testing headache: symbolic execution tools invent test cases to cover program branches, but carry so many settings that users fall back on poor defaults. Their Symetra system shows two linked overviews of how settings and setting groups change branch coverage. Testers can compare clusters of setups side by side and spot which choices drive coverage. In case studies, experts using the tool read setting effects, found complementary setups, and beat fully automated tuners on coverage and speed.
Catalog record
| CREATOR | D. Hong, M. Kim, S. Cha, and J. Jo Sungkyunkwan University, Republic of Korea |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56469 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.05349
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Symetra : Visual Analytics for the Parameter Tuning Process of Symbolic Execution Engines Volume: 45 Issue: 3 D. Hong, M. Kim, S. Cha, and J. Jo Sungkyunkwan University, Republic of Korea Abstract Symbolic execution engines such as KLEE automatically generate test cases to maximize branch coverage, but their numerous parameters make it difficult to understand the parameters’ impact, leading the user to rely on suboptimal default configurations. While automated tuners have shown promising results, they provide limited insights into why certain configurations work well, motivating the need for Human-in-the-Loop approaches. In this work, we present a visual analytics system, Symetra , designed to support Human-in-the-Loop parameter tuning of symbolic execution engines. To handle a large number of parameters and their configurations, we provide two complementary overviews of their impact on branch coverage values and patterns. Building on these overviews, our system enables collective analysis, allowing the user to contrast groups of configurations and identify differences that may affect branch coverage. We also report on case studies and a Human-in-the-Loop tuning process, demonstrating that experts not only interpreted parameter impacts and identified complementary configurations, but also improved upon fully automated approaches in both branch coverage and tuning efficiency. \SpecialIssuePaper \CGFccby ccs Human-centered computing Visual analytics ccs Software and its engineering Software testing and debugging † † year: 2026 † † year: 2026 † † editors: N. Andrienko, M. Chen, and B. Wang † † subject: Eurographics Conference on Visualization CGF Vol No † † orcid: 0009-0005-0890-2556 † † orcid: 0009-0004-7097-3171 † † orcid: 0000-0002-4697-8536 † † orcid: 0000-0002-5207-6010 1 Introduction One important research question in software engineering is how to ensure that a program is robust to arbitrary inputs. This is typically accomplished by generating test cases, executing them, and checking for uncaught exceptions or violated assertions. Symbolic execution [ 19 , 9 , 36 , 8 ] is a powerful method that automatically generates such test cases. The key idea is to systematically explore the program’s execution paths by treating input variables as symbolic values and generating inputs that exercise symbolic branches. Various symbolic execution engines, such as KLEE [ 8 ] , Crest [ 7 ] , or Sage [ 20 ] , have been researched and adopted. A critical challenge in effectively leveraging these engines lies in configuring the appropriate parameters that influence their behavior and thus branch coverage. Modern engines incorporate diverse, independent techniques, such as search strategies [ 12 , 22 ] and constraint solving [ 24 , 37 ] that come with numerous tunable parameters. As a result, it has ironically become more difficult to use the engines effectively due to the large number of parameters. For instance, KLEE [ 8 ] , a widely used symbolic execution engine, offers tens of tunable parameters spanning binary, continuous, and nominal types. This creates a complex, high-dimensional search space with non-linear interactions between parameters, making the tuning process challenging. Furthermore, a single configuration may not be optimal in terms of branch coverage; the user often requires a set of complementary configurations (i.e., configurations whose test cases cover different sets of branches) to further improve branch coverage. For example, a configuration with the Breadth-First Search (BFS) strategy may provide high overall branch coverage, while one with a Depth-First Search (DFS) strategy may better explore deeply nested branches. In this work, we aim to enable the user to effectively leverage symbolic execution engines by providing a visual analytics approach to the parameter tuning process. We observed that most users rely on command-line interfaces and general spreadsheet software to manually sift through thousands of configurations. Although automated tuners for the engines [ 22 , 12 , 13 ] have shown promising results, our domain analysis revealed four critical limitations of these fully automated approaches. First, the user rarely develops an understanding of why certain configurations work well, which would help them use the engines more efficiently in the future, such as when testing newer versions of the same software. Second, even though a small number of configurations can achieve high branch coverage if they target different parts of the program (i.e., complementary configurations), the discovery of such configurations occurs only by chance and is not well understood. Third, although existing tuners are designed to balance exploration and exploitation, the complex search space often leads to early saturation in branch coverage, i.e., repeatedly generating configurations that test similar code regions. Human guidance can help steer the search toward unexplored regions and break such saturation. Finally, certain configurations can result in testing failures, yielding zero branch coverage and wasting computational resources (e.g., running for several minutes without testing any branch). However, existing automated approaches do not provide meaningful insights into their causes. These limitations motivate Human-in-the-Loop approaches where visual analytics bridges automated optimization and human understanding. To tackle these challenges, we present Symetra , a visual analytics system that helps the user analyze and understand the parameter tuning process for symbolic execution engines. We demonstrate that the tuning process can be enhanced not only in a qualitative sense (i.e., providing insights that guide further improvements) but also in a quantitative sense (i.e., achieving higher branch coverage faster). To our knowledge, Symetra is the first visual analytics system designed specifically for the tuning process of symbolic execution engines, addressing a critical need in the software testing community where visual analytics has been largely absent. In collaboration with two domain experts, we identified user tasks and requirements that guided our design study. We validated our design through three complementary evaluations: qualitative case studies and expert interviews to assess the system’s analytical capabilities, and a quantitative evaluation demonstrating that Human-in-the-Loop parameter tuning improves fully automated approaches. The contributions of this paper are: [nosep] 1. Design and development of Symetra , the first visual analytics system for Human-in-the-Loop parameter tuning of symbolic execution engines, 2. Abstraction and characterization of the domain situation in symbolic execution-based software testing, and 3. Qualitative and quantitative evaluation of Symetra through case studies, expert interviews, and a quantitative demonstration of Human-in-the-Loop parameter tuning. Figure 1: Human-in-the-Loop Parameter Tuning. The gray box represents an experiment where a tuner iteratively optimizes configurations to maximize the branch coverage of the given program. The results of an experiment are trials, pairs of a configuration and its branch coverage, represented as branch coverage vectors ( V_{i} ) and branch coverage values ( C_{i} ), which are visualized in Symetra . The user then analyzes the results and refines the parameter space for subsequent experiments. 2 Related Work In this section, we survey relevant research and systems from both information visualization and software engineering. 2.1 Parameter Tuning for Symbolic Execution Parameter tuning in symbolic execution engines has been a critical challenge since they became a mainstream testing technique [ 6 , 4 , 25 ] . While symbolic execution engines such as KLEE [ 8 ] have become widely adopted in academia and industry [ 10 , 11 ] , maximizing their performance remains challenging due to the complex parameter space that must be tuned for each target program [ 27 , 38 , 41 ] . Early parameter tuning approaches were mostly manual and relied heavily on default settings. For instance, many researchers simply adopted KLEE’s