Hybrid Fault-Driven Mutation Testing for Python Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10
Saba Alimadadi, Golnaz Gharachorlu · 2026
Read Hybrid Fault-Driven Mutation Testing for Python Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10 by Saba Alimadadi, Golnaz Gharachorlu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19088.txt.
Introduction
Saba Alimadadi and Golnaz Gharachorlu blend two styles of mutation testing for Python code. Mutation testing grades a test suite by seeding small fake faults and seeing how many the tests catch. Old tools miss many fault types that are common in a loose typed language like Python.
Their hybrid method is driven by real fault patterns, not just simple code swaps. Python's dynamic typing breeds faults that stiff mutation rules never plant. A suite graded against realistic faults gives a truer read on whether the tests would catch the bugs that truly slip into Python projects.
Catalog record
| CREATOR | Saba Alimadadi, Golnaz Gharachorlu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-19088
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hybrid Fault-Driven Mutation Testing for Python Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3744916.3787804 ISBN: 979-8-4007-2025-3/2026/04 Saba Alimadadi, Golnaz Gharachorlu Abstract . Mutation testing is an effective technique for assessing the effectiveness of test suites by systematically injecting artificial faults into programs. However, existing mutation testing techniques fall short in capturing many types of common faults in dynamically-typed languages like Python. In this paper, we introduce a novel set of seven mutation operators that are inspired by prevalent anti-patterns in Python programs, designed to complement the existing general-purpose operators and broaden the spectrum of simulated faults. We propose a mutation testing technique that utilizes a hybrid of static and dynamic analyses to mutate Python programs based on these operators while minimizing equivalent mutants. We implement our approach in a tool called PyTation and evaluate it on 13 open-source Python applications. Our results show that PyTation generates mutants that complement those from general-purpose tools, exhibiting distinct behaviour under test execution and uncovering inadequacies in high-coverage test suites. We further demonstrate that PyTation produces a high proportion of unique mutants, a low cross-kill rate, and a low test overlap ratio relative to baseline tools, highlighting its novel fault model. PyTation also incurs few equivalent mutants, aided by dynamic analysis heuristics. † † cc-license: by 1. Introduction Software testing is a crucial phase in software development to ensure code quality. Mutation testing is a widely-studied technique for assessing the adequacy of a test suite ( 26 ; 27 ; 42 ; 47 ; 50 ; 55 ) . It is a fault-based approach that injects artificial faults, i.e. mutations , into the code using predefined mutation operators . A test suite is then exercised on the mutated code ( mutant ) to determine its effectiveness in detecting these faults. A mutant is considered killed if at least one test fails and survives otherwise. Surviving ( alive ) mutants reveal the inadequacies of tests in detecting the injected faults. Mutation testing quantifies the effectiveness of the test suite using the mutation score , the ratio of killed mutants to the total number of mutants. A high mutation score suggests a stronger test suite. However, a key challenge in mutation testing is the presence of equivalent mutants , which modify the code syntactically but do not affect its behaviour, making them undetectable by tests ( 42 ) . The presence of equivalent mutants can skew the mutation score and increase the cost of analysis. Various mutation testing techniques have been developed for different programming languages, including Java ( 5 ; 40 ; 56 ) , C/C++ ( 10 ; 12 ) , and JavaScript ( 37 ) . However, Python’s dynamic typing, flexible argument passing, and runtime attribute resolution create fault scenarios that are distinct from those in statically-typed languages. For example, Python allows removing required function arguments without triggering a compile-time error or accessing arbitrary attributes via reflection, making it easier for subtle faults to go unnoticed by general-purpose mutation operators. Common anti-patterns such as overly permissive function signatures, silent type coercions, and incomplete conditionals are frequent in Python codebases, yet difficult to catch with standard mutation techniques. This has led to the development of Python mutation testing tools in practice ( 1 ; 11 ) . 1 1 1 https://mutmut.readthedocs.io/en/latest/ 2 2 2 https://github.com/mutpy/mutpy While useful, these tools primarily apply general-purpose, syntax-based mutation operators and do not specifically target faults arising from Python’s dynamic features or commonly occurring anti-patterns. These existing tools simulate faults at the level of syntax and structure, such as deleting statements or altering arithmetic operators, but fail to capture the kinds of semantic or runtime-specific issues prevalent in Python codebases. Recent studies have analyzed frequent fault patterns in Python applications, highlighting limitations in current testing techniques and the need for more targeted approaches ( 62 ; 29 ) . Despite the prevalence of these faults and Python’s widespread adoption, 3 3 3 https://survey.stackoverflow.co/2023/ 4 4 4 https://madnight.github.io/githut/ 5 5 5 https://www.tiobe.com/tiobe-index/ there is currently no mutation testing technique that simultaneously targets Python-specific fault models and incorporates dynamic analysis to simulate and validate fault behaviors at runtime. In this paper, we introduce a novel fault-driven mutation testing technique for Python that leverages both static and dynamic analyses to identify and simulate Python-specific fault patterns. First, we discuss a set of common Python anti-patterns and introduce seven new mutation operators that mutate the code to artificially create these anti-patterns. Unlike prior techniques that operate purely statically, our dynamic instrumentation enables us to simulate faults that manifest only under specific runtime conditions, such as the removal of defaulted arguments or dynamic attribute lookups. To ensure practical relevance, our mutation operators are derived from prevalent, Python-specific bug/fix patterns categorized in a recent empirical study of 1,000 projects ( 29 ) . Then, we propose a technique based on a hybrid of static and dynamic analyses that mutates the code using our operators and applies a set of heuristics to reduce the number of equivalent mutants by eliminating those whose execution trace and state remain unchanged post-mutation. We implement our approach in a tool called PyTation and evaluate it using 13 open-source Python applications. We further analyze how the mutants generated by PyTation differ from those produced by a state-of-the-art Python mutation testing tool. In summary, our contributions are as follows. • A novel set of seven mutation operators derived from common Python anti-patterns, which broaden the mutation space and complement existing general-purpose mutation operators. • A hybrid static and dynamic mutation testing technique for generating these mutants, integrated with heuristics to reduce the number of equivalent mutants by observing runtime behaviour. • An open-source implementation of our approach in a tool called PyTation , along with all experimental data and scripts for reproducing our results, available at https://github.com/SEatSFU/pytation . • An empirical evaluation of PyTation on 13 real-world Python applications, demonstrating its ability to uncover inadequacies in Python tests. We also contextualize our technique by examining how it complements a state-of-the-art Python mutation testing tool. Our results show that PyTation identifies gaps in Python test suites by injecting realistic, Python-specific faults. It reveals inadequacies even in high-coverage code and produces a large proportion of unique mutants. While maintaining a low equivalent mutant rate, it complements general-purpose tools like Cosmic Ray by uncovering distinct behavioural faults and requiring different test coverage. Rather than replacing existing mutation testing methods, our technique augments them by exposing faults that general-purpose operators often miss, yielding deeper insight into test effectiveness. 2. Mutation Operators We introduce a set of seven Python-specific mutation operators, each designed to simulate the effects of a recurring anti-pattern frequently observed in real-world Python programs (summarized in Table 1 ). These anti-patterns represent recurring coding practices that reduce code reliability and are especially prevalent in dynamically-typed, idiomatic Python code. They are empirically grounded in a large-scale study ( 29 ) , which analyzed over 1,000 open-source projects and identified common single-statement bug/fix patterns. From these, we selected frequent Python-specific categories and designed operators that revert fixed code to