M ​ u ​ M ​ u ​ T ​ e ​ s ​ t ​ U ​ p MuMuTestUp : Mutation-based Multi-Agent Test Case Update DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06

Dawei Tian, Jiakun Liu*, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, Xiaohong Su · 2026

Read M ​ u ​ M ​ u ​ T ​ e ​ s ​ t ​ U ​ p MuMuTestUp : Mutation-based Multi-Agent Test Case Update DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Dawei Tian, Jiakun Liu*, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, Xiaohong Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19265.txt.

Introduction

Dawei Tian and colleagues tackle test cases that go stale as software changes fast under continuous delivery. Current language-model updaters chase runnable code and line coverage, yet they neglect whether assertions still catch faults. Their MuMuTestUp system splits the job across three agents covering mutation strength, uncovered lines and branches, and semantic retrieval for invented queries. Built on a 571-sample pull-request set from ten Java projects, it is tested with both open and closed leading models.

Catalog record

CREATORDawei Tian, Jiakun Liu*, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, Xiaohong Su
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68342
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19265

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

M ​ u ​ M ​ u ​ T ​ e ​ s ​ t ​ U ​ p MuMuTestUp : Mutation-based Multi-Agent Test Case Update DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Dawei Tian, Jiakun Liu*, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, Xiaohong Su Abstract . Modern software systems evolve rapidly under continuous integration and deployment (CI/CD) practices, in which tests act as critical gatekeepers of software quality. However, when substantial code changes are introduced, existing test cases may become obsolete, leading to compilation failures, erroneous test behaviors, or inadequate regression coverage. Such issues can disrupt CI/CD pipelines, degrade development productivity, and ultimately undermine overall software quality. Many efforts are devoted to designing automatic test case update methods to address these issues. The most recent approaches rely on large language models (LLMs) to iteratively refine test cases using execution feedback from compilation errors or coverage reports, and on context retrieved via exact-matching approaches. They also prioritize test executability and line coverage to quickly build executable, correct test cases from the original broken test cases. Despite their correctness, current approaches face three limitations: (1) they focus on executabilty but overlook the adequacy of test assertions, which lowers the capability of test cases to detect faults; (2) they utilize only coarse line coverage singals instead of specific information about uncovered lines and branches; (3) they use exact-matching context retrieval approaches, which fails to provide accurate context given potential hallucinated queries from LLMs. To address these challenges, we propose MuMuTestUp , a Mu tation-guided, Mu lti-agent framework for automated test case up dating. MuMuTestUp integrates three specialized agents: (1) a Mutation Analysis agent that leverages surviving mutants as indicators of weak or missing test assertions and generates individual repair instructions to strengthen or synthesize assertions for each surviving mutant, (2) a Coverage Analysis agent generates individual repair instructions for each uncovered line, uncovered branch rather than exposing raw coverage signals to the LLM, and (3) a Semantic Retrieval agent that uses semantic-similarity search to handle unavailable or hallucinated symbols. Additionally, we construct Prbench , a pull-request–level dataset of 571 samples from 10 open-source Java projects that considered cross-commit update scenarios, validated through three rounds of execution following prior studies to detect outdated tests. We evaluate MuMuTestUp against state-of-the-art baselines using both open-source and closed-source LLMs (Deepseek-V3.2 and GPT-4.1). With GPT-4.1, MuMuTestUp achieves a line coverage of 88.94%, branch coverage of 63.36%, and mutation score of 72.39%, outperforming the best baseline by 5.33%, 19.93%, and 16.66%, respectively. 1. Introduction With software development pipelines such as continuous integration and deployment (CI/CD), modern software systems evolve fastly ( 30 ; 29 ; 33 ) . Test cases play important roles in modern software development process since they validate and guarantee the correct functionality of software ( 35 ; 28 ; 17 ) . However, test cases can become outdated and broken with the rapid changes of requirements and code, which further leads to compilation failures, runtime errors, or missed regressions, threatening the software reliability and the efficiency of software development ( 19 ; 34 ) . To maintain software quality ( 26 ; 7 ; 14 ) , test cases must therefore be continuously co-evolved with production code to detect regressions as early as possible. Manually updating test cases to adapt code changes is time-consuming and error-prone ( 18 ; 38 ) , especially when test cases and code are often maintained by different teams in practice. Furthermore, software evolution is often complex, with individual updates spanning dozens or even hundreds of code changes. This makes comprehensive manual reasoning about test updates infeasible. Therefore, automating the evolution of outdated test cases is crucial for modern software development and maintanance. Early studies on automated test case update rely on program analysis techniques and heuristic rules ( 5 ; 4 ; 39 ; 25 ) , and they are inherently limited in handling the diverse update scenarios encountered in practice. With advances in machine learning, more recent work incorporates pre-trained language models for test case update ( 15 ; 31 ) . For example, TaRGET ( 31 ) employs CodeT5 to update outdated tests but is restricted to single-hunk changes. More recently, large language model (LLM)-based approaches are proposed for automatic test case update. For example, Chi et al. ( 1 ) introduce an LLM-based approach augmented with execution feedback and retrieval, Xu et al. ( 2 ) infer change intent to guide test updates, and Zhang et al. ( 42 ) proposed a post-hoc retrieval strategy to supply failure-related context. Despite the advances, existing research on automatic test case update has the following limitations. ➊Insufficient focus on test assertions. With prioritization on executability, existing approaches tend to generate weak assertions so the updated test cases can be successfully executed. However, test cases with weak assertions usually have low mutation scores ( 27 ; 3 ; 13 ) , indicating that they are not capable to discriminate between correct and abnormal behavior of code. Instead, they may always succeed in any programs. This further threatens the quality assurance capability of the entire test suite. ➋Inappropriate line coverage feedback. Existing approaches use line coverage as feedback, focusing on covering lines ( 2 ) or newly introduced code ( 1 ; 42 ) . However, line coverage measures only the syntactic execution of code and cannot guarantee that all logical branches are covered. Without branch coverage guidance, updated test cases may fail to validate important conditional logic, and thus they can reduce the effectiveness of regression testing ( 40 ; 37 ; 9 ) . ➌Limited context retrieval based on exact matching. Existing studies ( 1 ; 2 ; 15 ; 36 ) underestimated the importance of context when updating test cases. Context is essential in test case update as it provides necessary information about related code elements and repository knowledge. To date, only Synter ( 22 ) and TestUpdater ( 42 ) used language-server–based retrieval with exact matching. However, context retrieval methods based on exact matching can hardly handle the queries that contain hallucinated symbols generated by LLMs, limiting the robustness of test case updates in practice. To address these limitations, we propose MuMuTestUp , a Mu tation-guided Mu lti-agent-based Test Case Up date framework. Specifically, to address Limitation ➊, we design a Mutation Analysis agent, inspired by the mutation testing techniques ( 11 ; 10 ; 12 ; 6 ; 9 ; 24 ) that inject faults (mutants) into code to guide the generation of effective test assertions. It generates repair instructions for each surviving mutant, enabling the LLM to strengthen or generate strong assertions that specifically detect these mutants. To address Limitation ➋, we design a Coverage Analysis agent that generates repair instructions for each uncovered line and branch. This allows the LLM to update tests with precise guidance, avoiding blind exploration and effectively improving coverage. To address Limitation ➌, we design a Semantic Retrieval agent that retrieves necessary contextual symbols to guide test updates. To handle hallucinated or otherwise unavailable symbols generated by the LLM, the agent employs a vector database with semantic similarity matching, enabling robust

Read the complete text