Sustainability Analysis of Prompt Strategies for SLM-based Automated Test Generation Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; 9–12 June, 2026; Glasgow, Scotland, United Kingdom DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging CCS: Software and its engineering Automated software engineering CCS: Computing methodologies Artificial intelligence CCS: Hardware Power and energy
Pragati Kumari, Novarun Deb · 2026
Read Sustainability Analysis of Prompt Strategies for SLM-based Automated Test Generation Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; 9–12 June, 2026; Glasgow, Scotland, United Kingdom DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging CCS: Software and its engineering Automated software engineering CCS: Computing methodologies Artificial intelligence CCS: Hardware Power and energy by Pragati Kumari, Novarun Deb (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02761.txt.
Introduction
Kumari and Deb present the first systematic sustainability study of prompt strategies for small-language-model test generation. Seven prompt strategies across three open-source models are judged jointly on time, token use, energy, carbon, and coverage quality. Prompt choice has a large independent effect on sustainability, often outweighing model choice itself, with reasoning-heavy strategies costing far more than leaner ones for similar coverage.
Catalog record
| CREATOR | Pragati Kumari, Novarun Deb |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52534 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.02761
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sustainability Analysis of Prompt Strategies for SLM-based Automated Test Generation Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; 9–12 June, 2026; Glasgow, Scotland, United Kingdom DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software testing and debugging CCS: Software and its engineering Automated software engineering CCS: Computing methodologies Artificial intelligence CCS: Hardware Power and energy Pragati Kumari, Novarun Deb Abstract . The growing adoption of prompt-based automation in software testing raises important issues regarding its computational and environmental sustainability. Existing sustainability studies in AI-driven testing primarily focus on large language models, leaving the impact of prompt engineering strategies largely unexplored—particularly in the context of Small Language Models (SLMs). This gap is critical, as prompt design directly influences inference behavior, execution cost, and resource utilization, even when model size is fixed. To the best of our knowledge, this paper presents the first systematic sustainability evaluation of prompt engineering strategies for automated test generation using SLMs. We analyze seven prompt strategies across three open-source SLMs under a controlled experimental setup. Our evaluation jointly considers execution time, token usage, energy consumption, carbon emissions, and coverage test quality, the latter assessed through coverage analysis of the generated test scripts. The results show that prompt strategies have a substantial and independent impact on sustainability outcomes, often outweighing the effect of model choice. Reasoning-intensive strategies such as Chain of Thought and Self-Consistency achieve higher coverage but incur significantly higher execution time, energy consumption, and carbon emissions. In contrast, simpler strategies such as Zero-Shot and ReAct deliver competitive coverage test quality with markedly lower environmental cost, while Least-to-Most and Program of Thought offer balanced trade-offs. 1. Introduction Automated testing frameworks such as JUnit and Selenium, together with continuous integration and delivery pipelines, enable tests to be executed repeatedly across development cycles. While this automation improves software quality and development productivity, frequent test execution introduces non-trivial computational and energy costs, motivating increasing interest in sustainability-aware testing practices ( Verdecchia et al. 2021a ; Zaidman 2024 ) . Recent advances in large and small language models have transformed software testing by enabling automated unit test generation and augmentation. Empirical studies demonstrate that language models can generate useful unit tests under realistic conditions, although effectiveness varies across tasks, prompt designs, and execution settings ( Schaefer et al. 2024 ) . Hybrid approaches combining search-based testing with language model guidance show that interaction patterns and prompt structures can significantly influence coverage outcomes ( Lemieux et al. 2023 ) . At the same time, survey studies indicate that prompt engineering is pervasive in language model based testing workflows, while evaluation practices remain heterogeneous ( Wang et al. 2024 ) . Notwithstanding these advances, designing, validating, and refining correct and runnable test scripts continues to require substantial human effort. Despite growing evidence of effectiveness, the environmental impact of language model assisted testing remains less systematically studied. Recent work shows that even small language models can incur substantial energy overheads when used for automated unit test generation, underscoring the need to treat sustainability as a first-class evaluation dimension in testing workflows ( Durelli et al. 2025 ) . More broadly, emerging research on efficient and green language models for software engineering emphasizes the importance of balancing quality improvements with computational, energy, and carbon costs ( Shi et al. 2025 ) . However, task-specific analyses that jointly examine prompt strategies, sustainability metrics, and testing effectiveness are still limited. This paper investigates how prompt strategy selection influences sustainability and coverage test quality in automated unit test generation using small language models. We analyze three open-source SLMs deployed with 4-bit quantization to reflect resource-constrained inference settings commonly encountered in practice. Across these models, we systematically evaluate seven prompt strategies: Zero-shot ( Cheng et al. 2025 ; Della Porta et al. 2025 ) , Few-shot ( Tang et al. 2025 ) , Chain-of-Thought ( Wei et al. 2022 ) , Least-to-Most ( Zhou et al. 2023 ) , Program-of-Thought ( Payoungkhamdee et al. 2025 ) , Self-Consistency ( Wang et al. 2023 ; Pan and Zhang 2025 ) , and ReAct ( Yao et al. 2023 ) . For each model and prompt combination, we quantify total token usage (by aggregating input and output tokens), execution duration, CPU, GPU, and RAM energy consumption, and carbon emissions measured using CodeCarbon 1 1 1 https://codecarbon.io/ . Test effectiveness is assessed using normalized code coverage computed via coverage.py . The results show that prompt strategy selection can substantially influence tokenization behavior, runtime, energy consumption, and carbon emissions, even when the underlying model and task remain unchanged. Reasoning-intensive prompt strategies often improve coverage but incur disproportionately higher sustainability costs, whereas simpler or more structured prompts exhibit more favorable coverage quality–cost trade-offs. These observations highlight prompt design as an important and actionable factor in sustainability-aware software testing. Contributions. The main contributions of this paper are as follows: (I) We present a prompt-centric sustainability evaluation framework for automated unit test generation using small language models, integrating token usage, execution time, energy consumption, carbon emissions, and coverage test quality. (II) We apply this framework to carry out a controlled experimental evaluation of seven prompt strategies across three quantized open-source SLMs, enabling a systematic comparison of sustainability and coverage quality trade-offs. (III) We analyze the experimental results to examine how different prompt strategies influence execution behavior and sustainability-related metrics in automated testing. The remainder of this paper is organized as follows. Section 2 presents the related work. Section 3 describes the proposed framework. Section 4 details the experimental setup. Section 5 reports the results. Section 6 discusses the findings. Section 7 outlines threats to validity. Finally, Section 8 concludes the paper and discusses directions for future work. 2. Related Work Recent research has highlighted prompt engineering as a promising lever for reducing the environmental cost of language model inference, particularly in code generation and testing tasks. Prior studies systematically evaluate zero-shot, one-shot, and few-shot prompting variants, as well as custom prompt tags, using benchmarks such as CodeXGLUE ( Lu et al. 2021 ) and locally deployed models. These works measure execution time and energy consumption using tools like CodeCarbon and assess output quality via exact match and edit distance, demonstrating that carefully designed prompt structures can significantly reduce GPU energy usage without degrading generation quality ( Rubei et al. 2025 ) . Complementary analyses further reveal that non-semantic prompt characteristics, including code formatting elements such as indentation, whitespace, and newlines, can substantially inflate token counts and computational cost, motivating format-aware prompt optimization strategies ( Pan et al. 2025 ) . Building on prompt-structure analysis, recent work has examined how prompt linguistic complexity and interaction patterns influence energy–performance trade-offs in software engineering tasks. Green Prompt Engineering explicitly frames prompt design as a sustainability lever alongside hardware and model selection, showing that simpler, more readable prompts significantly reduce energy and carbon consumption with minimal loss in task performance ( De Martino et al. 2026 ) . Related profiling studies using fine-grained monitoring frameworks demonstrate that inference duration and response token length are strongly correlated with energy usage, while prompt linguistic complexity alone plays a comparatively smaller role ( Husom et al. 2024 ) . Empirical studies on automated unit test generation further show