TDFlow: Agentic Workflows for Test Driven Software Engineering
Kevin Han, Siddharth Maddikayala, Tim Knappe, Om Patel, Austen Liao, Amir Barati Farimani · 2025
Read TDFlow: Agentic Workflows for Test Driven Software Engineering by Kevin Han, Siddharth Maddikayala, Tim Knappe, Om Patel, Austen Liao, Amir Barati Farimani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23761.txt.
Introduction
Kevin Han and five co-authors present TDFlow, a workflow that aims coding agents at human-written tests and asks them to make the tests pass. Test-driven development frames repository-scale repair as a test-resolution task: the tests define success, and the agent must find the patch. Rather than one agent doing everything, TDFlow splits the work among four constrained sub-agents.
Sub-agents propose patches, debug them, revise them, and optionally write tests, so no single agent drowns in long context. Each part can then be improved on its own. Given human-written tests, TDFlow passed 88.8 percent of SWE-Bench Lite tasks, beating the next best system by nearly twenty-eight points, and 94.3 percent on SWE-Bench Verified. Hand checks of eight hundred runs found only seven cases of test hacking, counted as failures. The authors argue the remaining bottleneck is writing good reproduction tests.
Catalog record
| CREATOR | Kevin Han, Siddharth Maddikayala, Tim Knappe, Om Patel, Austen Liao, Amir Barati Farimani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-23761
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TDFlow: Agentic Workflows for Test Driven Software Engineering Kevin Han, Siddharth Maddikayala, Tim Knappe, Om Patel, Austen Liao, Amir Barati Farimani Abstract We introduce TDFlow, a novel test-driven agentic workflow that frames repository-scale software engineering as a test-resolution task, specifically designed to solve human-written tests. Given a set of tests, TDFlow repeatedly proposes, revises, and debugs repository-scale patches using precisely engineered sub-agents and tightly constrained tools. The workflow decomposes software engineering program repair into four components governed by respective sub-agents. This simple, forced decoupling of patch proposing, debugging, patch revision, and optional test generation (1) reduces long-context burden on any individual sub-agent, (2) focuses each sub-agent on specific, pre-defined sub-tasks, and (3) allows for specialized performance improvement on specific sub-tasks. When provided human-written tests, TDFlow attains 88.8% pass rate on SWE-Bench Lite (an absolute improvement of 27.8% over the next best baseline) and 94.3% on SWE-Bench Verified. Manual inspection of the 800 TDFlow runs within SWE-Bench Lite and Verified uncover only 7 instances of test hacking, which were subsequently counted as failures. Furthermore, we show that the primary obstacle to human-level software engineering performance lies within writing successful reproduction tests. We envision a human-LLM interactive system powered by TDFlow where human developers write tests solved by LLM systems. Together, these results indicate that modern LLMs, when embedded in a narrowly engineered, test-driven workflow, already achieve human-level test resolution – with the final frontier for fully autonomous repository repair being the accurate generation of valid reproduction tests. 1 Introduction Advancements in the code reasoning, long context, and agentic capabilities of large language models (LLMs) have led to the rise in LLM use cases as autonomous coders capable of fixing bugs at the repository scale ( Liu et al. 2024 ) . Models such as Claude 4 Sonnet, Kimi K2, Qwen3-Coder, and Gemini 2.5 Pro are provided with a set of file-interfacing tools and trained specifically on long-horizon reinforcement learning objectives ( Anthropic. 2025 ; Bai et al. 2025 ; Anil et al. 2023 ; Hui et al. 2024 ) . During inference time, these models are equipped with the same set of tools, provided an issue description and access to a repository, and are tasked with writing test cases, thinking of edge cases, and proposing a patch that addresses the issue. Afterwards, the patch is run against a series of hidden tests to check if the issue has been addressed. Current repository-scale software engineering benchmarks are focused on solving all hidden tests when provided with an issue description ( Jimenez et al. 2024 ; Zhang et al. 2025 ; Yang et al. 2024b ; Chowdhury et al. 2024 ) . However, in many real-world settings, software engineering and bug resolution is guided by the ethos of test driven development (TDD), where unit and integration tests are written before the actual code itself is implemented. The benefits of TDD include higher code quality, reduced recurring bug rates, and facilitate clearer, modular code that is easier to maintain and extend ( Cui 2024 ; Al-Saqqa et al. 2020 ) . On the other hand, the primary downside of TDD is the longer development time associated with writing tests for all features ( Nagappan et al. 2008 ) . We hypothesize that LLMs can streamline TDD by solving tests written by human developers . We envision a human-LLM collaborative software engineering system in which human guidance is provided to an LLM system via supplying ground-truth tests while LLM systems are relied upon to narrowly solve the tests and thereby expand the functionality of the repository. Figure 1: The workflow behind Test-Driven Flow (TDFlow). TDFlow is a purely test driven, agentic workflow for resolving repository scale issues. The entrypoint to TDFlow begins with either human-written reproduction tests or, optionally, to have TDFlow generate reproduction tests. Afterwards, the tests are run and provided to the Explore Files LLM sub-agent with the sole task of exploring the repository in order to propose a patch. The tests are run on the proposed patch before the Debug One sub-agent debugs each failing test individually with a dedicated debugger tool and generates reports. Those reports are used by the Explore Files sub-agent to propose another patch. In this work, we develop T est D riven Agentic Work flow (TDFlow), an agentic workflow specifically designed to solve human-written, ground-truth test cases at the repository scale. TDFlow consists of several LLM sub-agents operating within a precisely context-engineered environment. In the context of solving ground-truth test cases, workflows consisting of sub-agents pose several advantages over a single model, general-purpose, monolithic agent framework equipped with a broad set of tools. Primarily, monolithic LLM agents, or agents comprised of a single model and are indivisible in feature and capability with respect to each subtask, struggle with simultaneously decomposing and executing complex tasks into smaller subtasks ( Chen and Jiang 2025 ) . In TDD, once the ground-truth tests are written, the development workflow becomes 1) navigating the codebase to develop a solution to pass tests, 2) debugging any failed tests, and 3) repeating the process until all tests are resolved ( George and Williams 2003 ) . A strict workflow can explicitly model and enforce this decomposition of tasks, allowing each LLM sub-agent to focus on executing well-defined subtasks without the burden of additional planning. TDFlow is specifically tailored for resolving human-written test cases at the repository scale. On SWE-Bench Lite, when human-written, ground-truth tests are provided, the TDFlow system performs 27.8% better than the previous systems agentic software engineering systems. When generating its own tests, TDFlow achieves 69.8% performance on SWE-Bench Verified. However, when provided with human-written tests, TDFlow’s success rate reaches 94.3% – suggesting that TDFlow has achieved human-level performance in test resolution and can already be used in a human-LLM collaborative software engineering paradigm where humans write test cases which are solved by LLMs. We show that the final hurdle in human-level, fully autonomous software engineering with TDFlow lies within test generation, rather than issue resolution. Finally, we find a total of only 7 examples of test hacking after having three trained software engineers perform a manual analysis into all 300 SWE-Bench Lite and 500 SWE-Bench Verified human-written test runs. 2 Related Work 2.1 Test Driven Development Test driven development (TDD) has been repeatedly determined to significantly improve code quality at the expense of increased development time. An empirical study on four high impact teams at IBM and Microsoft found that TDD improved code quality, defined by defect density, by up to 90% but also increased development time by up to 35% ( Nagappan et al. 2008 ) . Another work split 24 professional engineers into 2 groups, one group developed with a TDD approach while the other group developed without a TDD approach ( George and Williams 2003 ) . Results showed that TDD led to 18% higher quality code defined by pass rate on functional black box tests. Similar to the findings of Nagappan et al. 2008 , the authors discover that TDD required 16% more development time. Overall, a large number of in-the-wild studies have been conducted both enforcing the efficacy as well as the additional time burden of TDD ( Williams et al. 2003 ; Cui 2024 ;