Grey-Box Fuzzing in Constrained Ultra-Large Systems: Lessons for SE Community
Jiazhao Yu, Yanlun Tu, Zhanlei Zhang, Tiehua Zhang, Cheng Xu, Weigang Wu, Hong Jin Kang, Xi Zheng · 2025
Read Grey-Box Fuzzing in Constrained Ultra-Large Systems: Lessons for SE Community by Jiazhao Yu, Yanlun Tu, Zhanlei Zhang, Tiehua Zhang, Cheng Xu, Weigang Wu, Hong Jin Kang, Xi Zheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10269.txt.
Introduction
Jiazhao Yu and co-authors share lessons from testing huge finance software in 2025. Fuzzing feeds odd inputs to software to shake out bugs. Giant service based systems block normal fuzzing through locked settings, tangled links, and strict security. The team built a tool called SandBoxFuzz for that hard setting.
The tool weaves test hooks into code and mines rules from how the system runs. It tracks coverage from logs inside the build line, with no live agents needed. Deployed at a major finance firm, it beat a leading general fuzzer on their services. The paper distills what worked for others facing giant locked systems.
Catalog record
| CREATOR | Jiazhao Yu, Yanlun Tu, Zhanlei Zhang, Tiehua Zhang, Cheng Xu, Weigang Wu, Hong Jin Kang, Xi Zheng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-10269
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Grey-Box Fuzzing in Constrained Ultra-Large Systems: Lessons for SE Community Jiazhao Yu, Yanlun Tu, Zhanlei Zhang, Tiehua Zhang, Cheng Xu, Weigang Wu, Hong Jin Kang, Xi Zheng Abstract . Testing ultra-large microservices-based FinTech systems presents significant challenges, including restricted access to production environments, complex dependencies, and stringent security constraints. We propose SandBoxFuzz , a scalable grey-box fuzzing technique that addresses these limitations by leveraging aspect-oriented programming and runtime reflection to enable dynamic specification mining, generating targeted inputs for constrained environments. SandBoxFuzz also introduces a log-based coverage mechanism, seamlessly integrated into the build pipeline, eliminating the need for runtime coverage agents that are often infeasible in industrial settings. SandBoxFuzz has been successfully deployed to Ant Group’s production line and, compared to an initial solution built on a state-of-the-art fuzzing framework, it demonstrates superior performance in their microservices software. SandBoxFuzz achieves a 7.5% increase in branch coverage, identifies 1,850 additional exceptions, and reduces setup time from hours to minutes, highlighting its effectiveness and practical utility in a real-world industrial environment. By open-sourcing SandBoxFuzz 1 1 1 https://zenodo.org/records/14679560 , we provide a practical and effective tool for researchers and practitioners to test large-scale microservices systems. Conference: 33rd ACM International Conference on the Foundations of Software Engineering; June 23–28, 2025; Trondheim, Norway 33rd ACM International Conference on the Foundations of Software Engineering (FSE Companion ’25), June 23–28, 2025, Trondheim, Norway DOI: 10.1145/3696630.3728546 ISBN: 979-8-4007-1276-0/2025/06 † † cc-license: by 1. Introduction The growing complexity of modern software systems has significantly increased the prevalence of software vulnerabilities, which can compromise operational efficiency and even lead to catastrophic failures, particularly in large-scale systems ( Lin et al., 2020 ) . Fuzz testing, or fuzzing, has emerged as a powerful automated technique for identifying erroneous behaviors in diverse software programs ( Miller et al., 1990 ; Böhme et al., 2016 ; Zhu et al., 2022 ; Yun et al., 2022 ; Zhang et al., 2021 ; Lipner, 2004 ) . Existing fuzzing tools have proven highly effective at detecting software defects in real-world applications ( Zalewski, 2025 ; Fioraldi et al., 2020 ; Project, 2025 ; Padhye et al., 2019b ) . By identifying and mitigating security vulnerabilities, fuzzing plays a crucial role in reducing their impact and minimizing long-term financial costs. Fuzzing has also gained significant traction within industry. Industrial studies ( Aizatsky et al., 2016 ; Liang et al., 2018 ) have developed powerful fuzzing tools capable of uncovering bugs in various programs (e.g., internal middleware) or accelerating the fuzzing process through parallel processing. However, the feasibility of applying fuzz testing within constrained industrial environments remains largely unexplored, especially where fuzzers can be difficult to integrate and scale, due to organization-wide security constraints, and the opacity and scale of the software systems. These unique issues have yet to be thoroughly discussed or addressed in existing research. Through our practical research and development experience in Ant Group, a prominent FinTech company, we observe three main challenges in deploying fuzz testing to large-scale FinTech systems: • Limited Permissions. Due to data security concerns, access to repositories and source code are restricted. Users of the fuzzers are not necessarily the authors of code in a target repository. They may not have full access to the repositories as well. Therefore, white-box-based fuzzing approaches ( Zhang et al., 2024 ; Arcuri et al., 2024 ; Gupta et al., 2024 ; Strässle, 2024 ; Kertusha, 2024 ; Liu et al., 2025 ; Yang et al., 2024 ; Arcuri et al., 2025 ; Zhang et al., 2025 ) that require access to source code are infeasible. Enabling the fuzzer users to rapidly set up a runnable fuzzing pipeline is a key factor for scalable deployment. • Restricted Test Environments. While performing testing within an enterprise’s settings, unlike operating in a fully controlled environment, test cases are enforced to execute using designated test frameworks and toolsets. These tools facilitate testing processes for developers. Due to their black-box nature, modifications to both the test environment and framework are strictly prohibited. While fuzzers rely on a tight feedback loop, the test environment restricts access to critical feedback, such as coverage information obtained through runtime agents. In Ant Group, the only permissible action is fetching logs from the test framework, which can only be done after the execution process of the entire test suite has been terminated. The existing corporate testing pipelines requires restarting the application for each fuzzing iteration, resulting in significant cold start overhead. This increases the challenges of integrating fuzz testing into industrial software systems. • Constraints from Input Constructions. Test inputs have various formats and complex compositions, which are introduced by designated test frameworks that are enforced by the company’s policies. File-based formats are ease-of-use and the most popular ones among developers. However, when combining with unique features of the test framework, additional overhead and constraints are introduced, including large-scale discard of inputs, limited mutation space, overhead of file operations and expensive manual effort on constructing mutation specifications. To tackle these challenges, in this paper, we propose SandBoxFuzz , a coverage-guided grey-box fuzzer for Java that bridges the gaps between generic fuzzing methodologies and practical deployment in industrial settings. Our improvements in SandBoxFuzz are based on three insights: First, the long startup cost of the application can be amortized by launching many test cases in one iteration. Second, bytecode can be augmented since it complies with data security regulations and produces useful metrics for guiding grey-box fuzzing, without breaking the black-box test environment. Third, substantial preparation time can be saved by replacing manual specification construction with automatic specification mining. As a result, rather than configure test input data and start the slow testing workflow, we utilize a sandbox approach to intercept inputs at specific pointcuts during the actual testing process, allowing us to bypass many of the restrictions imposed by existing test configurations. Specifically, we bootstrap the fuzzing process by launching the testing pipeline with duplicated valid inputs, while these dummy inputs will be replaced with real ones at runtime. Furthermore, bytecode of target program can be instrumented with print statements to track execution of each basic block in control-flow graph (CFG). Finally, we can save huge amount of manual effort, from hours to minutes, by mining specifications from the execution of the manually written test cases. Combining the observations and improvements, SandBoxFuzz makes fuzz testing deployable and scalable under the industrial context of Ant Group, with minimal manual effort. We apply SandBoxFuzz in Ant Group by integrating the fuzzing pipeline into its test environment. We evaluate SandBoxFuzz on target entry method s selected from several applications that are part of an ultra-large scale microservices system. Our experimental results show that SandBoxFuzz outperforms the manual testing approach at fuzzing effectiveness, and consumes less overall time including preparation and execution processes, with acceptable overhead introduced by its components. Moreover, SandBoxFuzz is better suited for an industrial environment where the testing framework is fixed and not optimized for fuzzing. It also stands out as a tool for users with limited knowledge of fuzzing to perform agile fuzz testing as it is the practical solution in terms of minimizing manual effort and scaling the fuzzing pipeline. In summary, this paper makes the following contributions: • We