MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering

Chuanzhe Guo, Jingjing Wu, Sijun He, Yang Chen, Zhaoqi Kuang, Shilong Fan, Bingjin Chen, Siqi Bao †, Jing Liu, Hua Wu, Qingfu Zhu, Wanxiang Che, Haifeng Wang · 2026

Read MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering by Chuanzhe Guo, Jingjing Wu, Sijun He, Yang Chen, Zhaoqi Kuang, Shilong Fan, Bingjin Chen, Siqi Bao †, Jing Liu, Hua Wu, Qingfu Zhu, Wanxiang Che, Haifeng Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22859.txt.

Introduction

Chuanzhe Guo and a large team present MEnvAgent, a system that builds test setups for coding tasks on its own. AI agents for software work need tasks that can be checked by running code. Building those runnable setups by hand, across many programming languages, is slow and scarce work.

MEnvAgent automates the construction of environments in many languages at once. That makes it possible to generate large numbers of tasks with known, checkable answers. Better task supply means coding agents can be trained and judged on firmer ground. Automated construction also keeps the environments verifiable, since each generated task can be executed and checked by design.

Catalog record

CREATORChuanzhe Guo, Jingjing Wu, Sijun He, Yang Chen, Zhaoqi Kuang, Shilong Fan, Bingjin Chen, Siqi Bao †, Jing Liu, Hua Wu, Qingfu Zhu, Wanxiang Che, Haifeng Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22859

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering Chuanzhe Guo, Jingjing Wu, Sijun He, Yang Chen, Zhaoqi Kuang, Shilong Fan, Bingjin Chen, Siqi Bao †, Jing Liu, Hua Wu, Qingfu Zhu, Wanxiang Che, Haifeng Wang Abstract The evolution of Large Language Model (LLM) agents for software engineering (SWE) is constrained by the scarcity of verifiable datasets, a bottleneck stemming from the complexity of constructing executable environments across diverse languages. To address this, we introduce MEnvAgent , a M ulti-language framework for automated Env ironment construction that facilitates scalable generation of verifiable task instances. MEnvAgent employs a multi-agent Planning-Execution-Verification architecture to autonomously resolve construction failures and integrates a novel Environment Reuse Mechanism that reduces computational overhead by incrementally patching historical environments. Evaluations on MEnvBench, a new benchmark comprising 1,000 tasks across 10 languages, demonstrate that MEnvAgent outperforms baselines, improving Fail-to-Pass (F2P) rates by 8.6% while reducing time costs by 43% . Additionally, we demonstrate the utility of MEnvAgent by constructing MEnvData-SWE, the largest open-source polyglot dataset of realistic verifiable Docker environments to date, alongside solution trajectories that enable consistent performance gains on SWE tasks across a wide range of models. Our code, benchmark, and dataset are available at GitHub . 1 Introduction The rapid evolution of Large Language Models (LLMs) has significantly advanced the exploration of repository-level code modification tasks within software engineering. Real-world issue resolution benchmarks, such as SWE-bench and its variants ( 13 ; 32 ; 33 ) , have emerged as the standard for evaluating the coding capabilities of LLMs. In these settings, autonomous agents like OpenHands ( 25 ) and SWE-Agent ( 29 ) are tasked with exploring repositories, localizing issues, generating patches (Pull Requests), and executing tests to validate solutions. Figure 1 : Comparison between manual environment construction and MEnvAgent (Ours). MEnvAgent leverages multi-agent collaboration to achieve automated environment construction, characterized by an efficient environment reuse mechanism. This execution-based verification is pivotal, not only for evaluation but also for emerging training paradigms like Reinforcement Learning with Verifiable Rewards (RLVR) ( 27 ) . However, the efficacy of such methods is constrained by the scalability of executable environment construction. Consequently, existing efforts face a dilemma: approaches based on static code metrics ( 28 ; 26 ) scale efficiently but provide only approximate verification signals, while manual construction ( 18 ) ensures quality but remains labor-intensive and largely restricted to Python. This leaves a critical gap for scalable, verifiable support across diverse programming languages. To bridge this gap, we introduce MEnvAgent , an automated framework engineered for scalable, polyglot environment construction (see Figure 1 ). Our approach aims to address two fundamental challenges in this field: (1) Complexity. Managing diverse dependencies across non-standard repositories requires deep expertise. Frequent construction failures (e.g., version conflicts, compilation errors) and inconsistent testing protocols (e.g., pytest or mvn test ) often lead to low success rates. (2) Time Consumption. The build process is inherently slow due to installation and compilation steps. Furthermore, environments are fragile; a single error often necessitates a costly “clean-slate” restart, creating a prohibitive overhead for large-scale data expansion. To tackle the complexity, we design a multi-agent architecture featuring an iterative Planning-Execution-Verification closed loop. Within this loop, specialized agents fulfill distinct responsibilities to iteratively diagnose and autonomously resolve construction failures to ensure high success rates. To address the time consumption, we propose a novel Environment Reuse Mechanism . Instead of building every instance from scratch, this mechanism retrieves similar historical environments and adapts them to the target repository snapshot by synthesizing and executing incremental environment patches. This approach avoids the heavy cost of full rebuilds, thereby boosting efficiency. Current environment construction benchmarks ( 16 ; 6 ; 9 ) are limited by narrow language coverage, non-executable evaluation, or insufficient quality assurance. To address these limitations and rigorously evaluate our approach, we construct MEnvBench , a comprehensive benchmark comprising 1,000 tasks across 10 languages, with strict execution-based evaluation and quality assurance. Extensive evaluations demonstrate that MEnvAgent outperforms state-of-the-art baselines across all languages, improving Fail-to-Pass (F2P) rates by 8.6% while reducing time costs by 43% . Furthermore, we leverage MEnvAgent to scale up verifiable data construction, yielding MEnvData-SWE , a realistic verifiable SWE training dataset. By fine-tuning open-source models on solution trajectories synthesized from this dataset, we achieve substantial performance gains on downstream SWE tasks, effectively validating the utility of MEnvAgent. The main contributions of this paper are as follows: • We introduce MEnvAgent , a multi-agent environment construction framework covering 10 programming languages, based on a Planning-Execution-Verification architecture. Notably, it incorporates a novel environment reuse mechanism that significantly reduces computational overhead. • We present MEnvBench , the first comprehensive benchmark for evaluating multi-language executable environment construction. This benchmark covers 10 mainstream languages across 200 open-source repositories, comprising a total of 1,000 tasks. • We release MEnvData-SWE , the largest open-source polyglot dataset of realistic verifiable Docker environments to date (see Table 12 ). Constructed via MEnvAgent, this dataset enables consistent performance gains for LLMs on downstream SWE tasks. 2 Problem Formulation In this section, we define the task of environment construction for verifiable SWE datasets. A verifiable task instance consists of two core components: Task Context and Executable Environment. The task context is gathered from GitHub, comprising the repository snapshot R and the issue with the related pull-request (PR). From the PR, we extract two distinct code changes: the fix patch representing logic modifications to resolve the issue, and the test patch containing new test cases to verify the fix. Let R_{fix} denote the repository state after applying the fix patch to R . Given a task context, the objective of environment construction is to determine a configuration triplet (B,\mathcal{P},T) . Here, B denotes the base image , \mathcal{P} represents the build process consisting of a sequence of installation commands, and T specifies the test configuration , which involves applying the test patch and executing the test command. The constructed environment is formally defined as S=\delta(B,\mathcal{P}) , where \delta represents the transition function. The fundamental goal is executability. Specifically, the constructed environment must allow repository state R_{fix} after applying the fix patch to pass the tests ( PASS ): \varepsilon(R_{fix},S,T)=0 (1) Here, \varepsilon(\cdot)\!=\!0 denotes that the tests are successfully passed. However, executability alone is insufficient. To ensure its validity as a verifiable environment, we enforce the Fail-to-Pass ( F2P ) criterion: \varepsilon(R,S,T)=1\quad\land\quad\varepsilon(R_{fix},S,T)=0 (2) This differential outcome guarantees that the environment accurately reproduces the specific issue (Fail) and verifies its resolution (Pass). (See Appendix B for further details). 3 MEnvAgent Design Figure 2 : Overview of MEnvAgent. (Top) The Environment Reuse Mechanism retrieves and adapts historical environments via incremental patching to reduce overhead. (Bottom) The Planning-Execution-Verification loop, where agents autonomously draft scripts, interactively repair build errors, and diagnose test failures to guide iterative refinement. In this section, we introduce the design of MEnvAgent, as illustrated in Figure 2 . We elaborate on two key components: the multi-agent architecture designed to construct executable environments and resolve construction failures, and the Environment Reuse Mechanism developed to accelerate this process by adapting historical environments (see Appendix C for specific details). 3.1 Multi-Agent Architecture The architecture of MEnvAgent is structured into three iterative stages: Planning, Execution, and Verification. Planning Stage. This stage is orchestrated by

Read the complete text