Call-Chain-Aware LLM-Based Test Generation for Java Projects DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging

Guancheng Wang, Qinghua Xu, Lionel C. Briand, Zhaoqiang Guo, Kui Liu · 2026

Read Call-Chain-Aware LLM-Based Test Generation for Java Projects DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging by Guancheng Wang, Qinghua Xu, Lionel C. Briand, Zhaoqiang Guo, Kui Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22046.txt.

Introduction

Wang and four colleagues present CAT, an LLM-based test generation approach that weaves call-chain and dependency context into prompts using static analysis. CAT tracks caller-callee links, object constructors, and third-party dependencies, and supports test fixing. The authors evaluate CAT on the Defects4J benchmark and four real-world GitHub projects posted after the LLM's cut-off date, comparing it with PANTA and EvoSuite. Across Defects4J projects, CAT lifts line, branch, and mutation scores by about 13, about 13, and 14.39 percentage points over PANTA, and also beats EvoSuite.

Catalog record

CREATORGuancheng Wang, Qinghua Xu, Lionel C. Briand, Zhaoqiang Guo, Kui Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78455
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22046

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Call-Chain-Aware LLM-Based Test Generation for Java Projects DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging Guancheng Wang, Qinghua Xu, Lionel C. Briand, Zhaoqiang Guo, Kui Liu Abstract . Large language models (LLMs) have recently shown strong potential for generating project-level unit tests. However, existing state-of-the-art approaches primarily rely on execution-path information to guide prompt construction, which is often insufficient for complex software systems with rich inter-class dependencies, deep call chains, and intricate object initialization requirements. In this paper, we present CAT, a novel call-chain-aware LLM-based test generation approach that explicitly incorporates call-chain and dependency contexts into prompts through dedicated static analysis. To construct executable, semantically valid test contexts, CAT systematically models caller–callee relationships, object constructors, and third-party dependencies, and supports iterative test fixing when generation failures occur. We evaluate CAT on the widely used Defects4J benchmark and on four real-world GitHub projects released after the LLM’s cut-off date. The results show that, across projects in Defects4J, CAT improves line and branch coverage by 18.04% and 21.74%, respectively, over the state-of-the-art approach PANTA, while consistently achieving superior performance on post-cutoff real-world projects. An ablation study further demonstrates the importance of call-chain and dependency contexts in CAT. 1. Introduction Unit tests play a fundamental role in ensuring software quality and reliability. However, writing high-quality unit tests remains labor-intensive and requires substantial domain knowledge, posing a long-standing challenge for software engineers ( 10 ; 35 ; 36 ; 26 ) . To mitigate this burden, a variety of automated test generation techniques have been proposed, including random testing (e.g., Randoop ( 26 ) ), search-based testing (e.g., EvoSuite ( 10 ) ), and symbolic execution–based approaches (e.g., KLEE ( 4 ) and Pex ( 35 ) ). These techniques have demonstrated effectiveness in detecting critical defects by systematically exploring program behaviors. More recently, advances in machine learning and large language models (LLMs) have further pushed the frontier of automated test generation ( 39 ; 27 ; 23 ) . LLM-based approaches leverage learned programming knowledge and contextual reasoning to generate readable, functional test cases, significantly improving both the automation and effectiveness of testing. Despite these advances, most existing works focus on method-level ( 38 ; 41 ; 31 ) or class-level ( 13 ; 44 ) test generation, which typically consider only the program logic within a single module. In contrast, project-level test generation, which must account for complex interprocedural dependencies, execution contexts, and interactions across multiple modules, remains largely underexplored and challenging. This gap limits the applicability of current techniques in large-scale software systems. State-of-the-art project-level test generation approaches leverage execution-path feedback to iteratively guide LLMs toward uncovered code regions, including PANTA ( 13 ) , PALM ( 40 ) , and SymPrompt ( 31 ) . These approaches dynamically collect uncovered execution paths or path constraints to guide test generation toward higher coverage. While such strategies have demonstrated promising results in certain settings, they rely on incorporating uncovered execution paths into the prompt as feedback signals and generally do not explicitly model dependencies across modules and execution contexts, as illustrated in Listing 1, where inter-class dependencies are required to correctly invoke the focal method but are not captured by existing methods. As a result, they often fail to generate valid tests for projects with complex inter-module dependencies. Empirical results in Section 4 further confirm this limitation. Using PANTA as a state-of-the-art representative of this line of work, we observe that on the JxPath project from the Defects4J ( 19 ) dataset–where methods heavily depend on external dependencies (e.g., dependencies on other modules, third-party libraries)–it achieves only 56.84% line coverage and 51.97% branch coverage. This observation suggests that the key difficulty lies not in identifying uncovered code locations but in constructing valid execution contexts, such as cross-class dependency setups, object initializations, and call sequences required to invoke the focal method. However, existing path-guided approaches provide limited insight into how such contexts are formed, as they focus primarily on uncovered code feedback without modeling dependency structures or object initialization processes. We further observe that the required execution contexts are inherently encoded in the call chains leading to the focal method, together with the constructors of the involved classes. These call chains capture how objects are created, how external dependencies are invoked, and the order in which different components interact. Therefore, we propose CAT , which is built upon a dedicated static analysis framework for systematically capturing relevant execution contexts. Specifically, CAT extracts call-chain information and associated initialization contexts, including caller-callee relationships, object constructions, and third-party dependencies, and explicitly provides the LLM with such rich project-level context, enabling it to generate valid tests even for methods with complex inter-module and third-party dependencies. Using the same benchmark (Defects4J) as in its original experiments, PANTA is, on average, outperformed by CAT by 18.04% and 21.74% in line and branch coverage, respectively. As discussed later in Section 4 , CAT generates tests for each focal method individually, whereas PANTA performs test generation once per focal class. As a result, CAT may incur higher overall generation costs due to its finer-grained generation process. Thus, we further perform a fairer comparison giving PANTA the same time budget as CAT , where the latter still achieves improvements of 12.24% in line coverage and 15.22% in branch coverage over the resulting time-bounded variant of PANTA . To further assess the generalizability of our approach and increase the realism of our comparisons, we evaluate CAT on four additional Java projects collected from GitHub that were released after the employed LLM became publicly available, thereby making them unlikely to have been part of its training data. As discussed in Section 4.1.1 , these projects are also more complex than those in Defects4J. On these unseen projects, CAT improves line and branch coverage by 25.09% and 25.91%, respectively, over the original PANTA , and by 13.02% and 14.68% over its time-bounded variant. These results suggest that explicitly modeling execution context, including call chains, constructor information, and third-party dependencies, substantially improves the LLM’s effectiveness for project-level test generation, particularly in previously unseen projects. In summary, this paper makes the following contributions: • We identify a key limitation of existing path-guided project-level test generation approaches: uncovered execution paths alone are insufficient for constructing valid execution contexts in projects with complex inter-module and third-party dependencies. To address this, we introduce a context modeling strategy and provide a dedicated static analysis framework to systematically capture relevant execution contexts, including call chains, object constructors, and third-party dependencies. • Based on this strategy, we propose CAT , an LLM-based project-level test generation approach. To the best of our knowledge, CAT is the first approach to explicitly incorporate call-chain- and dependency-aware execution contexts into LLM-based project-level test generation, enabling effective test generation for complex software systems. • We conduct an extensive empirical evaluation on Defects4J and four newly released GitHub projects that are unseen by the employed LLM and exhibit higher complexity than Defects4J in terms of inter-class dependencies and project structure, demonstrating substantial improvements over PANTA , the state-of-the-art baseline, and we release an open-source toolkit

Read the complete text