Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation (Experience Paper) DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 3–9, 2026; Oakland, California, United States Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’26), October 3–9, 2026, Oakland, California, United States
Chen Yang, Junjie Chen · 2026
Read Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation (Experience Paper) DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 3–9, 2026; Oakland, California, United States Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’26), October 3–9, 2026, Oakland, California, United States by Chen Yang, Junjie Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23509.txt.
Introduction
Yang and Chen present SeGa, a test-generation technique that hunts business-logic bugs by starting from product requirements rather than code alone. It groups requirements by business intent, derives detailed scenarios with preconditions and expected outcomes, and steers a language model to write tests from them. On four industrial projects it finds many more real bugs than rival methods, and deployment uncovered sixteen further bugs that developers confirmed and fixed.
Catalog record
| CREATOR | Chen Yang, Junjie Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71842 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23509
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation (Experience Paper) DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 3–9, 2026; Oakland, California, United States Proceedings of the 35th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’26), October 3–9, 2026, Oakland, California, United States Chen Yang, Junjie Chen Abstract . Business logic bugs violate intended business semantics and are particularly prevalent in enterprise software. Yet most existing unit test generation techniques are code-centric, making such bugs difficult to expose. We present SeGa, a semantics-driven unit test generation technique for uncovering business logic bugs. SeGa constructs a semantic knowledge base from product requirement documents, represented as a set of functionality entries that group related requirements under a common business intent. Given a focal method, SeGa retrieves the relevant functionality entries and derives fine-grained business scenarios with explicit preconditions, triggering actions, expected outcomes, and semantic constraints to guide LLM-based test generation. We evaluate SeGa on four industrial Go projects containing 60 real-world business logic bugs. SeGa detects 22 ∼ \sim 25 more bugs than four state-of-the-art LLM-based techniques and improves precision by 26.9% ∼ \sim 34.3%. Deployment across 6 production repositories further uncovers 16 previously unknown business logic bugs that were confirmed and fixed by developers, demonstrating SeGa’s practical value. From our industrial study, we summarize a series of lessons and suggestions for practical use and future research. 1. Introduction Automatic unit test generation plays a crucial role in modern software testing by enabling systematic and scalable validation of program behavior. A large body of prior work has studied Search-Based Software Testing (SBST), in which tests are generated or evolved to maximize structural coverage criteria such as statement, branch, or path coverage ( Fraser and Arcuri 2011 ; Pacheco and Ernst 2007 ; Lukasczyk and Fraser 2022 ) . More recently, advances in Large Language Models (LLMs) have enabled a new class of test generation techniques that produce unit tests directly from source code and its surrounding context, often achieving coverage comparable to or even exceeding that of traditional SBST approaches ( Yuan et al. 2023 ; Xie et al. 2023 ; Ryan et al. 2024 ; Wang et al. 2024 ; Yang et al. 2024a ; Yin et al. 2025 ) . Despite their differing technical foundations, both lines of work are fundamentally code-centric: they rely primarily on information extracted from the code itself to guide test generation. As a result, they are effective at uncovering defects that manifest in code structure or execution behavior, such as null-pointer dereferences ( Lin et al. 2021 ) , invalid operations ( Jang et al. 2025 ) , and type errors ( Yang et al. 2025b ) . However, by emphasizing how the code executes rather than what the software is intended to do, these techniques largely overlook intended business semantics defined by domain requirements. This limitation becomes particularly problematic in enterprise software, where correctness is governed by complex business policies, multi-step workflows, and legacy rules that are rarely made explicit in source code. Consequently, bugs arising from mismatches between implemented code and intended business semantics (referred to in this work as business logic bugs ) remain difficult to detect using existing code-centric unit test generation techniques. Incorporating business semantics beyond source code into unit test generation is therefore a promising direction for addressing such bugs. While both search-based and learning-based techniques could, in principle, benefit from richer semantic guidance, LLMs provide a particularly natural interface for consuming and reasoning over heterogeneous semantic information, which is often expressed in textual artifacts such as requirement documents. Accordingly, this work focuses on LLM-based unit test generation. However, enriching LLMs with external business semantics also introduces several challenges: Challenge 1: Business semantics extraction from informal specifications. The first challenge lies in reliably extracting high-quality business semantics from informal requirement artifacts. In practice, business semantics are primarily documented in product requirement documents (PRDs), which contain requirements about expected behavior, valid operational workflows, domain constraints, and implicit assumptions. However, PRDs are designed for human communication rather than automated analysis. They are often noisy and weakly structured, with core business semantics interleaved with background information, implementation notes, historical context, and change logs. Moreover, relevant semantic information is frequently scattered across multiple sections, expressed at different levels of abstraction, or embedded in non-textual formats such as tables or figures. These characteristics make it difficult to distill precise and actionable semantic representations that can serve as reliable guidance for unit test generation. Challenge 2: Effective utilization of extracted semantics for unit test generation. The second challenge concerns how to effectively utilize the extracted business semantics to guide the testing of a specific focal method. A single PRD typically covers a wide range of functionalities, only a small subset of which are relevant to the method under test. At the same time, the focal method often contains substantial implementation-specific logic (such as framework interactions, auxiliary computations, and defensive checks) that is orthogonal to the underlying business intent. This gap between document-level semantics and method-level code behaviors complicates the alignment of relevant semantics. Without effective utilization mechanisms, extracted semantics may be either too coarse-grained or misaligned with the focal method, limiting their effectiveness in generating realistic, targeted, and semantically meaningful unit tests. To address the two challenges, we propose SeGa ( Se mantics-driven unit test G ener a tion), a semantics-driven unit test generation approach that enhances LLM-based techniques by systematically integrating code context with business semantics extracted from requirement documents. First , SeGa constructs a semantic knowledge base from raw requirement documents. It leverages a multimodal LLM to convert tables and figures into textual form, filters out content unrelated to business semantics, and extracts functionalities that specify what the software is intended to do. Each extracted functionality is encoded as a structured domain-specific language (DSL) entry, which reduces ambiguity and supports efficient retrieval and downstream semantic reasoning during test generation. Second , given a focal method, SeGa retrieves method-relevant functionality entries from the semantic knowledge base to guide unit test generation. Specifically, a semantic reasoning agent analyzes the focal method together with its necessary surrounding code context and summarizes the method’s business intent as a concise natural-language description. This summary is then used as a query to retrieve relevant functionality entries. Because a retrieved functionality entry may aggregate multiple requirements, only some of which apply to the focal method, the semantic reasoning agent derives fine-grained business scenarios that isolate the requirement(s) relevant to the focal method and encode them in a scenario DSL. Each scenario is represented using explicit preconditions , triggering actions , expected outcomes , and constraints , thereby providing precise and focused semantic context for subsequent unit test generation. Finally , a test generation agent jointly reasons over the semantic context and the code context to generate unit tests . To ensure practical applicability in real-world projects with strict build and dependency constraints, SeGa incorporates a standalone compilation-repair component that automatically fixes compilation issues in generated tests, thereby improving their executability. We evaluated SeGa on four large-scale, production Go projects developed and maintained by a leading international IT company. These projects are actively used in industrial settings and