Automatically Generating Single-Responsibility Unit Tests

Geraldine Galindo-Gutierrez · 2025

Read Automatically Generating Single-Responsibility Unit Tests by Geraldine Galindo-Gutierrez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06431.txt.

Introduction

Geraldine Galindo-Gutierrez maps a thesis on machine-made code tests that each check one thing, in a plan from 2025. Test bots save hours and catch faults, but they chase breadth: calls pile up to raise the span, and the point of each test blurs. Coders shun such tests as hard to read.

The plan gives made tests a one-duty frame up front, so the star method of each test stands clear. Sundry frames will be bred and bred-tested for span, fault catch, and ease of read. Coders will grade the read in the flesh. The bet: form lifts trust and read ease at no real cost to catch. If the bet pays, made tests may earn a seat in real code shops.

Catalog record

CREATORGeraldine Galindo-Gutierrez
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE11 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-06431

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automatically Generating Single-Responsibility Unit Tests Geraldine Galindo-Gutierrez Abstract Automatic test generation aims to save developers time and effort by producing test suites with reasonably high coverage and fault detection. However, the focus of search-based generation tools in maximizing coverage leaves other properties, such as test quality, coincidental. The evidence shows that developers remain skeptical of using generated tests as they face understandability challenges. Generated tests do not follow a defined structure while evolving, which can result in tests that contain method calls to improve coverage but lack a clear relation to the generated assertions. In my doctoral research, I aim to investigate the effects of providing a pre-process structure to the generated tests, based on the single-responsibility principle to favor the identification of the focal method under test. To achieve this, we propose to implement different test representations for evolution and evaluate their impact on coverage, fault detection, and understandability. We hypothesize that improving the structure of generated tests will report positive effects on the tests’ understandability without significantly affecting the effectiveness. We aim to conduct a quantitative analysis of this proposed approach as well as a developer evaluation of the understandability of these tests. This paper has been accepted for publication at the 47th International Conference on Software Engineering (ICSE 2025). The final version will appear in the official conference proceedings published by ACM/IEEE. I Problem Statement Unit tests act as the first line of prevention against potential bugs introduced during software evolution. When a unit test fails, it is required that developers fully understand the behavior under test before any fix. When reading a unit test, developers expect to easily detect the faulty behavior and pinpoint the issue. For this reason, several principles and good practices are suggested for creating readable, maintainable, and reliable tests [ 1 , 2 ] . For example, consider testDepositToAccount in Figure 1 which contains a descriptive name (a readability good practice) and the main method under test is clearly stated (single-responsibility principle). ⬇ 1 public void testDepositToAccount () { 2 BankAccount account = new BankAccount ( "X" , 100.00); 3 account . deposit(50.00) ; 4 assertEquals (150.00, account . getBalance (),0.01); 5 } Fig. 1: Test for deposit method in BankAccount class. Despite its advantages, test writing is a complex and time-consuming process for developers [ 3 , 4 ] . Automatic test generation tools address this issue by producing test suites with optimized coverage and limited human intervention [ 5 , 6 , 7 , 8 ] . However, their focus on coverage and mutation score leaves other properties, such as readability and maintainability, coincidental [ 9 , 10 , 11 , 12 ] . Studies show that generated tests present quality pitfalls [ 13 , 14 , 15 ] and Shamshiri et al. found that developers are skeptical of using generated tests due to their unclear purpose [ 16 ] . To exemplify this issue, consider test17 in Figure 2 . This test invokes three methods from BankAccount class: closeAccount , deposit , and transferFunds ; and verifies their effects by checking the account balance. However, it is unclear which of these is the focal method, the behavior under test, as all method calls can be separately verified by the same assertion in line 6. In case of failure, it is not easy to determine which method is not working as expected as the test seems to be checking various methods, thereby violating the single-responsibility principle. ⬇ 1 public void test17 () throws Throwable { 2 BankAccount bankAccount0 = new BankAccount ( "" , 0.0); 3 bankAccount0 . closeAccount (); 4 bankAccount0 . deposit (665.49); 5 bankAccount0 . transferFunds ( bankAccount0 , 0.05); 6 assertEquals (665.49, bankAccount0 . getBalance (),0.01); 7 } Fig. 2: Generated test for BankAccount class. Different test generation tools fall short to follow the single-responsibility principle for a non-trivial portion of their tests. In consequence, developers have to spend more time understanding the purpose of generated tests before using them. Various approaches have previously attempted to address this problem using post-processing approaches such as minimizing tests to avoid redundancies [ 17 ] , improving readability [ 18 , 19 , 20 ] , or adding test documentation [ 21 ] . Other approaches improve test quality during search, for example, integrating metrics to avoid test smells during generation [ 22 ] or using readability metrics and models [ 18 ] . However, these approaches are inherently limited by the current functioning of generation tools, where tests are mainly optimized for coverage and do not consider a defined structure. In consequence, the generated test code is often perceived as machine generated [ 10 , 18 , 23 ] , which influences the adoption of test generation tools in industry. II Proposed Approach Unlike previous approaches, we propose to address this problem by changing the underlying foundations of test generation to produce tests that follow the single responsibility principle by construction in order to improve their understandability for developers. The common representation of a test T in state-of-the-art search-based generation tools is a list of statements [s_{1},\ldots,s_{n}] of variable length n where each s has a defined statement type (primitive, constructor, field, method, assignment) [ 17 , 8 ] . The n statements participate in crossover and mutation operations focusing on optimizing a coverage-guided fitness function. After reaching 100% coverage or exhausting the search budget, the tools generate assertions and apply different post-processing steps [ 22 ] . However, not considering structural properties may cause multiple methods invoked in the same test increasing coverage, but hindering the purpose of the test. Test Representation using Focal Methods. We propose to provide test cases with a defined structure that indicates the focal method under test from construction. Previous studies have used focal methods when generating tests using LLM-based approaches [ 24 , 23 , 25 ] , identifying the focal method in the input. However, there are several considerations to adapt this approach in search-based tools, we must evaluate different structures and their effects in mutation and crossover operations. For example, a possible representation could be to fix the last statement of the test, s_{n} , as the focal method and consider only the n-1 statements for crossover and mutation operations. To follow single-responsibility principle, the proposed structures are focused on identifying the statements dedicated to the test setup (e.g., object constructor and method invocations) and the statements that contain the tested behavior. Assertion Generation for Single-Responsibility Tests. In an initial study, we observed that in a portion of generated tests, the assertions are not clearly related to the invoked methods of the class under test [ 14 ] . Assertion generation is currently based on mutation analysis, where assertions that uniquely detect at least one new mutant are kept as part of the generated test. However, not all newly detected mutants are necessarily related to the main behavior under test. We propose to adapt assertion generation to our proposed structures and maintain only assertions related to the focal method of the test. With this approach, we aim at grouping assertions based on the method their detected mutants are related to. This process can allow new refactorizations [ 13 ]

Read the complete text