TDAD: Test-Driven Agentic Development – Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis
Pepe Alonso, Sergio Yovine, Victor A. Braberman · 2026
Read TDAD: Test-Driven Agentic Development – Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis by Pepe Alonso, Sergio Yovine, Victor A. Braberman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17973.txt.
Introduction
Alonso, Yovine, and Braberman present TDAD, an open tool that warns coding agents which tests a patch may break. It maps links between source code and tests before a change is made. On SWE-bench Verified, TDAD cut regressions from 6.08% to 1.82% in their tests. Plain test-driven prompts without this map made regressions worse. As an agent skill, TDAD also raised the fix rate from 24% to 32%.
Catalog record
| CREATOR | Pepe Alonso, Sergio Yovine, Victor A. Braberman |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36609 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17973
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TDAD: Test-Driven Agentic Development – Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis Pepe Alonso, Sergio Yovine, Victor A. Braberman Abstract . AI coding agents can resolve real-world software issues, yet they frequently introduce regressions —breaking tests that previously passed. Current benchmarks focus almost exclusively on resolution rate, leaving regression behavior under-studied. This work develops a method to enhance agents with appropriate structural knowledge that leads to meaningful regression reduction while improving resolution rate, and presents TDAD (Test-Driven Agentic Development), an open-source tool that performs pre-change impact analysis for AI coding agents. TDAD builds a dependency map between source code and tests so that before committing a patch, the agent knows which tests to verify and can self-correct. The map is delivered as a lightweight agent skill—a static text file the agent queries at runtime. Evaluated on SWE-bench Verified with two open-weight models running on consumer hardware (Qwen3-Coder 30B, 100 instances; Qwen3.5-35B-A3B, 25 instances), TDAD reduced regressions by 70% (6.08% → \to 1.82%) compared to a vanilla baseline. In contrast, adding TDD procedural instructions without targeted test context increased regressions to 9.94%—worse than no intervention at all. When deployed as an agent skill with a different model and framework, TDAD improved issue-resolution rate from 24% to 32% , confirming that surfacing contextual information outperforms prescribing procedural workflows. All code, data, and logs are publicly available. 1. Introduction Large-language-model (LLM) based coding agents have demonstrated impressive ability to resolve real-world software issues. On SWE-bench Verified ( jimenez2024swebench ) , state-of-the-art agents resolve over 70% of GitHub issues from popular open-source repositories. Evaluation, however, centers almost exclusively on resolution rate : the fraction of issues for which the agent’s patch passes the issue-specific tests. A complementary question— how many previously-passing tests does the patch break? —receives far less attention, despite recent evidence that regressions and CI/CD failures are a leading cause of rejected agent-authored pull requests ( metr2026swebench ; ehsani2026agentfail ) . Regressions are hard to prevent without dependency awareness. An agent can either run all tests (too slow for large codebases) or run only tests near the changed files (missing indirect dependencies). This problem parallels classical regression test selection (RTS) ( elbaum2014techniques ; legunsen2016extensive ) , but the agentic context introduces novel requirements: changes are generated programmatically, the agent has a limited context window, and test verification must happen before submission rather than in a CI pipeline afterward. In the baseline experiments, a vanilla agent caused 562 pass-to-pass (P2P) test failures across 100 instances—an average of 6.5 broken tests per generated patch. The SWE-bench evaluation harness already collects the data needed to measure regressions: the PASS_TO_PASS test set records tests that passed before the gold patch and should still pass afterward. Yet this data is not surfaced in leaderboard rankings. Recent evidence reinforces this concern: METR ( metr2026swebench ) found that roughly half of SWE-bench-passing patches would not be merged by real maintainers, and Ehsani et al. ( ehsani2026agentfail ) show that CI/CD failures are a leading cause of rejected agent-authored pull requests. Regression rate deserves first-class status alongside resolution rate. This paper presents TDAD ( T est- D riven A gentic D evelopment), an open-source tool that performs pre-change impact analysis for AI coding agents. TDAD builds a dependency map between source code and tests so that before committing a patch, the agent knows exactly which tests to verify and can self-correct (Figure 1 ). The key insight is that agents do not need to be told how to do TDD; they need to be told which tests to check . At runtime the agent needs only grep and pytest —no graph database, MCP server, or API calls. The evaluation spans two phases on SWE-bench Verified—a controlled comparison on 100 instances with a single model, followed by a generalization study with a different model and agent framework—yielding four key findings: (1) 70% regression reduction. In Phase 1 (100 instances, Qwen3-Coder 30B, single-agent setup), TDAD reduced test-level regression rate from 6.08% to 1.82% (562 \to 155 P2P failures). (2) TDD Prompting Paradox. In the same Phase 1 setup, an ablation adding TDD procedural instructions (write tests first, then implement) without telling the agent which specific tests to check actually increased regressions to 9.94%—worse than vanilla. Procedural instructions without targeted context are counterproductive. (3) Resolution improvement as agent skill. In Phase 2 (25 instances, different model Qwen3.5-35B-A3B + OpenCode agent), TDAD improved resolution from 24% to 32% and generation from 40% to 68%, confirming generalization beyond the original setup. (4) Autonomous self-improvement. An auto-improvement loop (10-instance subset) iteratively refined TDAD ’s own configuration, raising resolution from 12% to 60% with 0% regression. Read issue Plan fix Implement TDAD: at-risk tests Verify tests Submit patch fix regressions Figure 1. Agentic workflow with TDAD . After implementing changes, the agent queries TDAD ’s test map to identify at-risk tests, verifies them, and self-corrects if regressions are detected. The dashed arrow represents the self-correction loop. Contributions. (1) An open-source tool for graph-based test impact analysis ( pip install tdad ); (2) a benchmark methodology elevating regression rate to a first-class metric; (3) empirical evidence across two models and agent frameworks; (4) an autonomous auto-improvement loop for iterative tool refinement; (5) all code, data, and 28 experiments released under MIT license. 2. Related Work 2.1. AI Coding Agents and Benchmarks SWE-bench ( jimenez2024swebench ) evaluates agents on resolving GitHub issues from 12 popular Python repositories, establishing the primary benchmark for AI coding agents. The SWE-bench Verified subset provides human-validated ground truth for 500 instances. SWE-Agent ( yang2024sweagent ) introduces an agent–computer interface optimized for repository navigation, achieving strong results through careful tool design. AutoCodeRover ( zhang2024autocoderover ) combines code search with spectrum-based fault localization to improve issue resolution. OpenHands ( wang2024openhands ) provides a unified platform for developing and evaluating coding agents across multiple benchmarks. The benchmark ecosystem continues to evolve. SWE-smith ( yang2025swesmith ) scales training data by automatically synthesizing task instances from arbitrary Python codebases, producing 50k instances and training open-source models that achieve 40.2% on SWE-bench Verified. SWE-Bench++ ( wang2025swebenchpp ) extends evaluation to 11 programming languages with 11k instances. SWE-CI ( chen2026sweci ) shifts focus from one-shot bug fixing to long-term codebase maintenance via continuous integration loops, requiring agents to handle sequences of commits over months. All of these efforts focus primarily on resolution rate; regression behavior is reported only incidentally, if at all. The SWE-bench harness does execute P2P tests, but results are not surfaced in leaderboard rankings. Ehsani et al. ( ehsani2026agentfail ) study 33k agent-authored pull requests on GitHub and find that CI/CD pipeline failures and regression are among the most common rejection reasons, confirming that resolution alone is an insufficient measure. This work argues that this omission creates a perverse incentive that rewards aggressive patching regardless of side effects. 2.2. Regression Testing Regression test selection (RTS) and prioritization have a long history in software engineering. Elbaum et al. ( elbaum2014techniques ) survey techniques for improving regression testing in CI environments, establishing that even simple selection strategies can dramatically reduce test execution time. Legunsen et al. ( legunsen2016extensive ) evaluate static RTS at scale, finding that class-level dependency tracking achieves good precision. Gligoric et al. (