SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing?

Noam Tarshish, Nofar Selouk, Daniel Hodisan, Bar Ezra Gafniel, Yuval Elovici, Asaf Shabtai, Eliya Nachmani · 2026

Read SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing? by Noam Tarshish, Nofar Selouk, Daniel Hodisan, Bar Ezra Gafniel, Yuval Elovici, Asaf Shabtai, Eliya Nachmani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25737.txt.

Introduction

Tarshish and six colleagues propose SAFEdit, a multi-agent system for instructed code editing that splits the work into specialized roles. A planner drafts an explicit edit plan, an editor applies minimal literal changes, and a verifier runs real tests. When tests fail, a failure abstraction layer turns raw logs into structured feedback for another editing round. On the EditBench benchmark of 445 editing instances in five languages, SAFEdit reached 68.6 percent task success. That beat the single-model baseline and a ReAct single-agent baseline.

Catalog record

CREATORNoam Tarshish, Nofar Selouk, Daniel Hodisan, Bar Ezra Gafniel, Yuval Elovici, Asaf Shabtai, Eliya Nachmani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47945
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25737

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing? Noam Tarshish, Nofar Selouk, Daniel Hodisan, Bar Ezra Gafniel, Yuval Elovici, Asaf Shabtai, Eliya Nachmani Abstract . Instructed code editing is a significant challenge for large language models (LLMs). On the EditBench benchmark, 39 of 40 evaluated models obtain a task success rate (TSR) below 60%, highlighting a gap between general code generation and the ability to perform instruction-driven editing under executable test constraints. To address this, we propose SAFEdit , a multi-agent framework for instructed code editing that decomposes the editing process into specialized roles to improve reliability and reduce unintended code changes. A Planner Agent produces an explicit, visibility-aware edit plan, an Editor Agent applies minimal, literal code modifications, and a Verifier Agent executes real test runs. When tests fail, SAFEdit uses a Failure Abstraction Layer (FAL) as part of the framework to transform raw test logs into structured diagnostic feedback, which is fed back to the Editor to support iterative refinement. We compare SAFEdit against both prior single-model results reported for EditBench and an implemented ReAct single-agent baseline under the same evaluation conditions. We used EditBench to evaluate SAFEdit on 445 code editing instances in five languages (English, Polish, Spanish, Chinese, and Russian) under varying spatial context variants. SAFEdit achieved 68.6% TSR, outperforming the single-model baseline by +3.8 percentage points and the ReAct single-agent baseline by +8.6 percentage points. The iterative refinement loop was found to contribute +17.4 percentage points to SAFEdit’s overall success rate. SAFEdit’s automated error analysis further indicates a reduction in instruction-level hallucinations compared to single-agent approaches, providing an additional framework component for interpreting failures beyond pass/fail outcomes. 1. Introduction Software developers spend a large amount of time making targeted modifications to existing code: fixing bugs, adding parameters, refactoring functions, and extending classes to support new behaviors. As a result, automated assistance for editing existing code has become a practical and high-impact LLM application ( 6 ; 8 ) . Unlike generative coding tasks, instructed code editing requires precise reasoning over existing structure, preservation of invariants, and minimal, targeted modifications that satisfy explicit correctness constraints ( 6 ; 8 ; 4 ) . Recent evaluations of LLMs point to a concerning reality: even cutting-edge models struggle to reliably perform instructed code editing. On the EditBench benchmark ( 6 ) , a collection of 540 real-world editing tasks drawn from developer Integrated Development Environment (IDE) sessions and validated by executable test suites, 39 of 40 evaluated models were found to obtain a task success rate (TSR) of less than a 60%. The best single-model baseline (claude-sonnet-4) achieved a TSR of just 64.8% under the most informative context variant. These results indicate that model capability alone is insufficient. Trustworthy code editing requires structured reasoning about the intended modification, careful restriction of changes to the relevant regions, and validation through actual test execution. Recent studies have demonstrated the ability of multi-agent architectures to perform complex coding tasks by decomposing them into specialized sub-tasks ( 9 ; 5 ) . However, existing approaches focus primarily on code generation or full-issue resolution ( 11 ) , with no methods specifically designed for or evaluated on the structured, context-sensitive, instructed editing setting. This paper presents SAFEdit ( S tructured A gentic F ramework for Trustworthy Code Edit ing), a multi-agent system that decomposes instructed code editing into three sub-tasks that are executed by specialized agents orchestrated via the CrewAI framework ( 2 ) : a Planner that translates the instruction into a structured, visibility-aware edit plan without generating any code, an Editor that performs controlled modification by applying the plan minimally and literally, and a Verifier that performs execution-based validation via real test runs in a sandboxed environment. When tests fail (not all of the unit tests passed), a failure abstraction layer (FAL) transforms raw test logs into structured diagnostic feedback that identifies the failed test, exception type, expected vs. actual values, and a suggested repair action, which is passed back to the Editor; a maximum of three refinement iterations are permitted. We evaluated SAFEdit on 445 code editing instances in five languages (English, Polish, Spanish, Chinese, and Russian), using the same EditBench benchmark visibility variants and test infrastructure to ensure comparability with the original EditBench benchmark. SAFEdit achieves an average TSR of 68.6% across all five languages, surpassing the ReAct single-agent baseline by +8.6pp on average and the best single-model baseline (claude-sonnet-4, 64.8%) by +3.8pp . Our main contributions are summarized as follows: • Architectural Decomposition for Instructed Code Editing. We introduce SAFEdit, a structured agentic framework that divides the instructed code editing task into three sub-tasks: planning, editing, and verification, which are performed by three specialized agents. Our results show that this architectural structure yields consistent TSR improvements over strong single-agent and LLM baselines while holding the backbone model constant, suggesting that structured decomposition, together with execution-grounded iterative refinement, contributes meaningfully to performance beyond backbone choice alone. • Robustness to Context Exposure. Across all evaluated languages, both SAFEdit and ReAct exhibit a systematic decline in the TSR as context exposure is reduced, consistent with the intuition that less contextual information makes the task harder. This contrasts with the original EditBench findings, where context-level effects were model-dependent and often inconsistent, suggesting that our multi-language evaluation reveals a more stable and interpretable signal. • Iterative Verification as a Structural Mechanism. We demonstrate that verification-driven refinement is not merely corrective but structurally transformative, contributing on average +17.4 percentage points over first-pass performance and eliminating regression errors observed in the centralized reasoning baselines. • Failure Redistribution Through Role Separation. Using an error taxonomy, we show that SAFEdit reshapes the distribution of failure categories: reducing regression errors entirely and shifting failures from instruction-level hallucination toward implementation-level refinement gaps, revealing qualitative differences in reasoning behavior beyond aggregate success rates. 2. Related Work 2.1. Instructed Code Editing The instructed code editing task is fundamentally distinct from traditional code generation: models must interpret natural-language instructions to modify existing code, rather than synthesize new programs from scratch ( 3 ) . In the traditional setting, widely used evaluations of code-focused LLMs, including the Codex benchmark suite, assess a model’s ability to perform function-level generation, while subsequent program-synthesis studies often assessed performance in isolated settings without prior editing history ( 4 ; 1 ) . This generation-centric evaluation paradigm does not reflect how developers interact with code assistants in practice, where the dominant workflow is to iteratively edit and refine existing files in response to natural-language instructions. More recent benchmarks such as SWE-bench evaluate whether models can resolve real GitHub issues end-to-end; doing so requires repository-level reasoning, including navigating multiple files and using large code contexts during inference ( 11 ) . While this setting is realistic and challenging, it targets full issue resolution at the repository level rather than localized edits to a given snippet under a concrete instruction. In contrast, instructed code editing focuses on such localized, instruction-driven changes within the context of an existing file, which is precisely the scenario captured by EditBench and our proposed SAFEdit framework ( 6 ) . In addition to existing general code-generation benchmarks, several datasets were designed specifically to evaluate instructed code editing. Can It Edit? serves

Read the complete text