RePro: Leveraging Large Language Models for Semi-Automated Reproduction of Networking Research Results

Yining Jiang, Wenyun Xu , Qingyu Song, Yuling Lin, Xuanhao Liu, Xiaoqiang Zheng, Qiang Su, Lizhao You, Lu Tang, Wanjin Feng, Linghe Kong, Qiao Xiang , Jiwu Shu · 2025

Read RePro: Leveraging Large Language Models for Semi-Automated Reproduction of Networking Research Results by Yining Jiang, Wenyun Xu , Qingyu Song, Yuling Lin, Xuanhao Liu, Xiaoqiang Zheng, Qiang Su, Lizhao You, Lu Tang, Wanjin Feng, Linghe Kong, Qiao Xiang , Jiwu Shu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21074.txt.

Introduction

Reproducing a networking paper's results is notoriously hard, since code is often missing and systems are intricate. Yining Jiang and sixteen co-authors present RepLLM in a 2025 paper for SIGCOMM 2026. The framework uses language models to rebuild implementations from the papers themselves.

Four specialized agents divide the work. One parses the paper's content, one designs the architecture, one generates code, and one audits and repairs the result. A shared coordination structure keeps the agents consistent across long contexts and logical dependencies. Ordinary code generation frameworks stumble on exactly those features of network systems. RepLLM aims to make reproduction a routine, automated step, strengthening the credibility of networking research whose claims others must build on.

Catalog record

CREATORYining Jiang, Wenyun Xu , Qingyu Song, Yuling Lin, Xuanhao Liu, Xiaoqiang Zheng, Qiang Su, Lizhao You, Lu Tang, Wanjin Feng, Linghe Kong, Qiao Xiang , Jiwu Shu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-21074

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RePro: Leveraging Large Language Models for Semi-Automated Reproduction of Networking Research Results Yining Jiang, Wenyun Xu , Qingyu Song, Yuling Lin, Xuanhao Liu, Xiaoqiang Zheng, Qiang Su, Lizhao You, Lu Tang, Wanjin Feng, Linghe Kong, Qiao Xiang , Jiwu Shu Abstract Reproducing networking research is a critical but challenging task due to the scarcity of open-source code. While Large Language Models (LLMs) can automate code generation, current approaches lack the generalizability required for the diverse networking field. To address this, we propose RePro, a semi-automated reproduction framework that leverages advanced prompt engineering to reproduce network systems from their research papers. RePro combines few-shot in-context learning with Structured and Semantic Chain of Thought (SCoT/SeCoT) techniques to systematically translate a paper’s description into an optimized, executable implementation. The framework operates through a three-stage pipeline: system description extraction, structural code generation, and code optimization. Our evaluation with five state-of-the-art LLMs across diverse network sub-domains demonstrates that RePro significantly reduces reproduction time compared to manual efforts while achieving comparable system performance, validating its effectiveness and efficiency. 1 Introduction The study of network systems spans protocols, distributed systems, and large-scale network architectures, among other areas, and its high complexity makes reproduction highly challenging. Reproducing results of networking research is of substantial value to both academia and industry. Reproduction is essential for ensuring the reliability and reproducibility of results, which strengthens academic credibility and provides industry with evidence-based guidance for technology selection and system optimization handigol2012reproducible . Moreover, it allows researchers to acquire a deep understanding of system design and implementation, while honing engineering and analytical skills yan2017learning . Despite its significance, reproduction remains challenging due to the limited availability of source code and the effort required to faithfully re-implement complex systems, underscoring the need for tools and methods that facilitate efficient and accurate replication. Current reproduction methods include: (1) running publicly available prototypes provided by authors yang2015real or recreated by others zhang2020apkeep ; (2) manually re-implementing systems from paper descriptions. The availability of public code for networking research is limited. For instance, even in top-tier conferences such as SIGCOMM and NSDI, only 40.15% of papers published in the last decade have been accompanied by open-source implementations. Unofficial, third-party reproductions are even scarcer snlabflash , and private code sharing is often precluded by factors such as patents or commercial interests. Manual re-implementation, while insightful, is a laborious process requiring significant expertise—a challenge that grows with the proliferation of complex networking research. This difficulty is in stark contrast to fields like machine learning, where a novice can often replicate a paper in about a week yan2017learning . The substantially higher time investment required for networking systems creates a significant barrier to entry and slows the overall pace of innovation. To reduce reproduction costs and accelerate research, recent studies have investigated the use of large language models (LLMs) for automated code generation, including code completion, debugging, and synthesis github_copilot_2025 ; openai-chatgpt ; zhang2024pydex ; rahmani2021multi ; yen2021semi . In particular, recent advances in paper-to-code generation seo2025paper2code ; lin2025autop2c ; luo2025intention have shown promising results in machine learning and biomedical domains, where standardized frameworks (e.g., PyTorch pytorch , TensorFlow tensorflow ) and widely adopted benchmarks facilitate reproducibility. Networking research presents the following challenges: heterogeneous architectures, complex paper parsing, and diverse programming languages. Existing work xiang2023toward relies on prompt-heavy methods requiring hundreds of manual interactions, imposing substantial reproduction costs; wang2025llm targets only satellite networks, limiting its applicability; and chen2022software ; yen2021semi focus solely on well-formatted inputs within narrow domains, making them unsuitable for broader networking contexts. These limitations underscore the need for a systematic method to parse, reasoning, and generate code from complex, unstructured networking research with LLMs. To address this critical issue, we propose RePro, a semi-automated framework that leverages advanced LLMs and structured prompt engineering to streamline the paper reproduction process, thereby reducing the manual labor and time cost associated with reproduction while maintaining high fidelity to the original paper’s design and implementation details. We design a four-step systematic workflow to reproduce the system in a network research. Step ①: System Description Extraction . RePro automatically decomposes the system’s architecture. First, we preprocess the paper to perform structured text analysis and process multimodal information from the source document. This process extracts foundational system information, such as its sub-field, core problem, and high-level components. Next, drawing upon principles of software engineering modularity, we define clear evaluation metrics for the system architecture. We design a series of specialized prompt templates to guide an LLM in generating a well-defined system architecture composed of multiple functional modules. Step ②: Framework-Level Code Generation . RePro translates the extracted system architecture into a modular code framework. This process integrates principles from program analysis with advanced prompt engineering techniques, notably Structured Chain-of-Thought (SCoT) and In-Context Learning (ICL). This methodology guides the Large Language Model (LLM) to first comprehend the context of each functional module and then reason progressively, culminating in the generation of specific structural, functional, and procedural requirements. To enhance correctness, a strong correlation and an explicit mapping are established between the content of the source paper and the generated code framework. This mapping provides precise guidance for the subsequent code population phase. Step ③: Function-Level Code Generation . To automate code population, RePro transforms the previously generated framework into executable code. For each function, we employ a few-shot learning approach, providing the LLM with examples that map requirement descriptions to their corresponding code implementations. We apply the Semantic Chain-of-Thought (SeCoT) method, which incorporates control and data flow, to structure the model’s output. This approach ensures logical rigor and verifies correctness by compelling the LLM to explicitly trace data manipulation and program execution. Step ④: Code Optimization . RePro is designed for the automated correction and refinement of the generated code. To resolve syntactic or runtime errors, we employ a feedback loop where the erroneous code and its corresponding compiler or runtime error report are provided as input to an LLM. The LLM is then instructed to perform a detailed analysis of the error report—including stack traces, exception types, and specific messages—to identify the root cause and generate a corrected version of the code. This iterative debugging mechanism resolves immediate errors and serves to optimize the framework’s overall generation strategy by learning from failure cases, thereby enhancing its robustness and accuracy. The main contributions of this paper are as follows. • Systematic Prompt Engineering. We propose a systematic prompt engineering strategy that integrates few-shot learning, structured chain-of-thought reasoning, and semantic chain-of-thought reasoning. This unified design enables comprehensive paper analysis, automated module framework generation, and iterative code completion and optimization, ultimately supporting the construction of complete, executable systems. • First Semi-Automated Framework for Network Research Reproduction. We address the novel and critical challenge of efficient paper reproduction in computer networking by proposing RePro, the first semi-automated reproduction framework based on LLM prompt engineering. Through its two key components—automated system architecture generation and semi-automated module code generation—RePro systematically transforms complex network research papers into complete, executable systems, significantly enhancing reproduction efficiency. • Reliable and Fast Reproduction. We validate the effectiveness and generality of the RePro framework through a comprehensive evaluation. Using five different LLMs, we conduct reproduction experiments on nine distinct network systems. Our results show

Read the complete text