RAT: RunAnyThing via Fully Automated Environment Configuration

Renhong Huang, Dongdong Hua, Yifei Sun, Sitao Ding, Hanyang Yuan, Daixin Wang, Yang Yang † · 2026

Read RAT: RunAnyThing via Fully Automated Environment Configuration by Renhong Huang, Dongdong Hua, Yifei Sun, Sitao Ding, Hanyang Yuan, Daixin Wang, Yang Yang † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23190.txt.

Introduction

Huang and six colleagues automate environment setup for repository-level coding tasks, a bottleneck that slows autonomous code agents. Current tools lean on preset artifacts or cover only some languages, limiting them on diverse real repos. Their framework, RAT, runs a multi-stage pipeline with language-aware abstraction, image setup, special config tools, and a sandbox, working across languages on any repo. They also build RATBench for testing, and RAT beats strong baselines by 36.1 percent on environment setup success on average.

Catalog record

CREATORRenhong Huang, Dongdong Hua, Yifei Sun, Sitao Ding, Hanyang Yuan, Daixin Wang, Yang Yang †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69494
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23190

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RAT: RunAnyThing via Fully Automated Environment Configuration Renhong Huang, Dongdong Hua, Yifei Sun, Sitao Ding, Hanyang Yuan, Daixin Wang, Yang Yang † Abstract Automating repository-level software engineering tasks is a foundational challenge for autonomous code agents, largely due to the difficulty of configuring executable environments. However, manual configuration remains a labor-intensive bottleneck, necessitating a transition toward fully automated environment configuration. Existing approaches often rely on pre-defined artifacts or are restricted to specific programming languages, limiting their applicability to real-world repositories. In this paper, we first propose RAT (RunAnyThing), a language-agnostic framework for automated environment configuration on arbitrary repositories. RAT features a multi-stage pipeline that integrates semantic initialization, a planning mechanism, specialized toolset, and a robust sandbox for configuration. Furthermore, to enable rigorous evaluation, we propose RATBench , a benchmark that reflects the the distribution and heterogeneity of real-world repositories. Extensive experiments demonstrate that RAT achieves state-of-the-art performance, improving the Environment Setup Success Rate (ESSR) by an average of 29.6% over strong baselines. 1 Introduction The evolution of Large Language Models (LLMs) has shifted the frontier of autonomous programming from simple snippet generation ( 26 ; 1 ; 5 ) to complex, repository-level engineering ( 25 ; 10 ; 18 ; 20 ) . However, unlike code snippets, repository-level tasks demand strict adherence to intricate inter-dependencies and environment-specific configurations. Without an executable environment, even logically correct code remains unverifiable and functionally invalid. Consequently, environment configuration has emerged as a key bottleneck in the development of autonomous agents ( 9 ) . Moreover, environment configuration is not merely a matter of software convenience, but a fundamental requirement for the development of code LLMs ( 13 ; 12 ; 14 ) . Generally, automated environment configuration plays three critical roles: (1) Scalable Data Synthesis : It enables scalable benchmark construction by transforming static repositories into verifiable datasets, rather than relying on manually curated benchmarks such as SWE-bench ( 10 ) . In addition, t helps synthesize accurate execution traces that are essential for LLM training ( 6 ) . (2) Execution-based Reinforcement : It supports functional feedback loops, allowing reward models to move beyond static or heuristic signals ( 12 ) toward true executability. (3) System Reliability : It ensures deployment reproducibility, overcoming the inherent brittleness of CI/CD integration. Overall, automated environment configuration is key to transforming code agents from symbolic generators into reliable autonomous systems. Although several pioneering efforts have attempted to automate environment setup, they often impose constraints that limit generalizability. For instance, INSTALLAMATIC ( 16 ) and EXECUTIONAGENT ( 2 ) rely heavily on pre-existing artifacts, such as curated Dockerfiles, installation contexts, or CI logs. While Repo2Run ( 9 ) introduces a dual-environment architecture to decouple configuration from monitoring, it still operates within a rigid framework that lacks the flexibility to handle diverse, uncurated repositories. Collectively, these methods are ill-suited for scaling to thousands of real-world repositories due to their dependence on specific prior knowledge and limited language support. At a fundamental level, the generalizability of environment configuration is constrained by information sparsity and intricate cross-component dependencies that these rule-based frameworks struggle to address. To overcome the limitations of existing approaches, we introduce RAT (RunAnyThing), a modular, language-agnostic framework for fully automated environment configuration. RAT employs an LLM-driven multi-stage pipeline starting with ImageRetriever, which semantically analyzes repositories to select optimal base images and reduce configuration overhead. A dual-mode planning mechanism enables handling of complex or previously unseen repositories, while a specialized toolset and long-term expertise accumulation resolve execution ambiguities and support cross-session knowledge retention. Together, these components provide scalable, adaptive, and robust environment configuration for diverse repositories. Furthermore, to rigorously evaluate environment configuration methods under realistic repository settings, we introduce RATBench, a large-scale multilingual benchmark comprising over 2,000 GitHub repositories. Unlike existing datasets that are limited in language coverage or biased toward trivial projects, RATBench is constructed via stratified sampling to capture real-world diversity in project distribution, programming languages, and availability, and is validated through a rigorous executability-driven pipeline. Extensive experiments on RATBench demonstrate that RAT achieves a state-of-the-art Environment Setup Success Rate (ESSR) and exhibits environment configuration capabilities that surpass human experts. 2 Preliminary Configuration Artifacts. We refer to files that specify environment setup as configuration artifacts . Typical artifacts include Dockerfiles, CI pipelines, build manifests (e.g., package.json , pom.xml , Cargo.toml ), lockfiles (e.g., poetry.lock ), and explicit test scripts. When present, these artifacts partially or fully define the required runtime environment (e.g., language and library versions) and the verification sequence. When artifacts are absent or incomplete, both would be inferred from documentation (e.g., README ) and repository structure. Environment Configuration. Let \mathcal{R} denote a repository comprising source code and metadata, and let \mathcal{E}(\mathcal{R}) denote the set of feasible containerized environments compatible with \mathcal{R} . The environment configuration is to construct an environment e\in\mathcal{E}(\mathcal{R}) along with a verification sequence \pi(\mathcal{R}) , where \pi(\mathcal{R}) specifies the repository-dependent execution procedure (e.g., tests or build commands). A configuration is considered successful if the execution of \pi(\mathcal{R}) within e terminates without error. The output of the task is either a runnable container image or a Dockerfile that deterministically builds such an image. Execution Trace. An environment configuration session induces an interaction trace \tau(\mathcal{R})=((a_{1},o_{1}),\ldots,(a_{T},o_{T})) , where each action a_{t} is a tool invocation and each observation o_{t} is the resulting system feedback. The trace represents the agent’s interaction history with the environment and serves as the basis for subsequent decision making. 3 Framework: RAT Figure 1 : The architecture of RAT (RunAnyThing). The framework consists of several primary modules: (1) Language-Agnostic Abstraction, which identifies project languages and encapsulates domain-specific protocols into a unified interface; (2) ImageRetriever, which performs semantic analysis of the repository to select optimal base images; (3) Agent Planning, featuring both a fixed Standard Plan Mode and an adaptive Automated Plan Mode using a structured document for long-term reasoning; and (4) Specialized Toolset, which provides integrated tools for file manipulation, error recovery, and verification within a robust sandbox. As shown in Figure 1 , we introduce RAT ( R un A ny T hing), a modular, language-agnostic framework for the automated configuration of complex software environments. To achieve these objectives, RAT employs a multi-stage pipeline that conducts LLM-driven agents within containerized sandboxes, leveraging an extensive suite of specialized tools. We next describe the framework in detail. Language-Agnostic Abstraction. At the onset of environment configuration, RAT first comprehends the entire project and identifies the corresponding programming languages. It encapsulates language-specific heuristics, including dependency patterns (e.g., pom.xml for Java, Cargo.toml for Rust), package manager protocols, and specialized test runners. This unified interface allows the model to invoke language-specific toolsets through a unified protocol. As repositories often contain multiple programming languages, we currently identify the language with the largest proportion as the primary language. ImageRetriever for Initialization. Environment configuration can leverage existing environments to improve success rate, particularly for complex environments that share common architectural patterns with others. Given the repository, RAT first establishes the Language-Agnostic Abstraction module. Then, the ImageRetriever module conducts an LLM-based semantic analysis of the repository’s documents and configuration artifacts to determine the optimal executable environment, including language versions, OS variants, deep learning framework, critical dependencies, etc. Based on the analysis, the ImageRetriever module first infers a standard base image

Read the complete text