StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
Xinkui Zhao, Yifan Zhang, Zhengyi Zhou, Yueshen Xu · 2025
Read StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution by Xinkui Zhao, Yifan Zhang, Zhengyi Zhou, Yueshen Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11665.txt.
Introduction
Xinkui Zhao and three co-authors introduce StackPilot, a system that checks computer code written by smart models without needing normal compilers. Testing generated code is hard because each language needs its own tools and setup. StackPilot instead treats each function in the code as its own agent that can reason and cooperate.
A second idea lets the language model itself act as the runner, steered by a stack based schedule. Snapshots save the full state at each step, so switching context loses nothing. In tests the system reached reliability rates between eighty-nine and ninety-seven percent. That beat baseline methods across many tasks and languages. The authors pitch it as a scalable checker for machine written code.
Catalog record
| CREATOR | Xinkui Zhao, Yifan Zhang, Zhengyi Zhou, Yueshen Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-11665
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution Xinkui Zhao, Yifan Zhang, Zhengyi Zhou, Yueshen Xu Abstract Recent advances in large language models (LLMs) have substantially enhanced automated code generation across a wide range of programming languages. Nonetheless, verifying the correctness and executability of LLM-generated code remains a significant challenge, as traditional methods rely on language-specific compilers and environment-dependent runtimes. To overcome these limitations, we introduce StackPilot, an LLM-native, multi-agent framework designed for language-agnostic code verification and execution, which operates independently of conventional toolchains. StackPilot offers three principal innovations: (1) a Function-as-Agents paradigm, in which each function is modeled as an autonomous agent capable of fine-grained reasoning and collaborative verification; (2) an LLM-as-Executor strategy, which enables scalable verification via stack-based scheduling; and (3) a novel snapshot mechanism that preserves complete execution contexts, facilitating deterministic and lossless context switching during verification. Empirical evaluations demonstrate that StackPilot achieves framework reliability rates between 89% and 97%, substantially outperforming baseline approaches. These results indicate that StackPilot can reliably verify and execute a significantly larger proportion of LLM-generated code across diverse programming tasks compared to existing methods. Introduction The rapid advancement of large language models (LLMs) has significantly expanded their capabilities across a wide range of domains, including natural language processing ( Achiam et al. 2023 ; Touvron et al. 2023 ; Liu et al. 2024a ) and automated code generation ( Guo et al. 2024 ; Li et al. 2023b ; Zhu et al. 2024 ; Chowdhery et al. 2023 ; Fried et al. 2022 ) . These developments have facilitated widespread adoption of LLMs in software development workflows ( Dong et al. 2024 ; Jiang, Wang, and Wang 2023 ; Nijkamp et al. 2022 ) , with tools such as GitHub Copilot ( Dakhel et al. 2023 ) and Cursor increasingly automating coding tasks, thereby enhancing development efficiency and reducing routine manual labor. Despite these advancements, LLM-generated code frequently exhibits several critical shortcomings. First, it often omits essential robustness checks ( Liu et al. 2023 ) —such as null checks, boundary conditions, and value validations—leading to increased risk of runtime errors ( Li et al. 2025 ) . Second, necessary error-handling constructs (e.g., try-catch blocks) are commonly missing, resulting in unhandled exceptions ( Liu et al. 2024b ) . Third, conditional expressions may be incorrect or inconsistent with those found in human-written code, introducing subtle logic bugs ( Zhong and Wang 2024 ) . Current verification approaches primarily rely on automated compilation and unit testing against benchmark datasets ( Jiang et al. 2024 ; Chen et al. 2024a ; Chen et al. 2021 ; Hendrycks et al. 2021 ; Haque et al. 2023 ) . While this framework enables large-scale evaluation by first checking syntactic validity then assessing functional correctness through test cases, it has significant limitations. The approach depends entirely on code executability, requiring specialized environments and toolchains. Furthermore, error messages from failed compilation or tests are often poorly interpretable by LLMs, hindering effective automated feedback and improvement ( Zhong and Wang 2024 ) . To address these challenges, we propose StackPilot , an LLM-native framework grounded in the novel Function-as-Agents paradigm. In this approach, each program function is modeled as an autonomous agent, enabling fine-grained context management and function-level reasoning. Distinct from traditional methods that rely on language-specific compilers and runtimes, StackPilot leverages LLMs not only for code generation, but also as code executors and environment simulators. This design enables direct interpretation and reasoning about program behavior, providing language-agnostic and scalable verification capabilities. At the core of StackPilot’s verification pipeline is a dynamically assembled team of verification agents, each assigned to a specific function within the program. These agents perform function-level reasoning, collaboratively propagate verification states, and resolve dependencies across the program’s call graph. This agent-based architecture enables modular verification, facilitates integration with external tools, and supports comprehensive program-level verification—all without requiring native runtime environments. A central feature of StackPilot is its stack-based scheduling mechanism, inspired by conventional function call stacks. This mechanism orchestrates agent interactions and information flow, allowing verification agents to invoke one another, maintain precise execution contexts, and collaboratively resolve complex inter-function dependencies. During scheduling, StackPilot maintains agent snapshots to capture and restore the complete execution context of each function call, ensuring deterministic and lossless context switching throughout the verification process. We comprehensively evaluate StackPilot by integrating it with several code generation models and testing them on the Human-Eval ( Chen et al. 2021 ) and MBPP ( Austin et al. 2021 ) . Our empirical results show that StackPilot achieves a framework reliability of from 89% to 97%, substantially outperforming baseline methods and robustly verifying and executing a much greater proportion of LLM-generated code across diverse programming tasks. These results demonstrate that StackPilot is a highly reliable, plug-and-play framework that requires no environment configuration, enabling seamless verification of code in a wide variety of settings. The main contributions of this work are as follows: • We present StackPilot , an LLM-native, plug-and-play framework for code verification and execution that operates independently of traditional toolchains or environment configuration. • StackPilot is built upon three key principles— Function-as-Agents , LLM-as-Executor , and stack-based scheduling with agent snapshots —which together enable unified management and execution of function-level code. • Through extensive experiments on the Human-Eval and MBPP benchmarks with multiple code generation models, we demonstrate that StackPilot achieves over 89% framework reliability, substantially surpassing baseline methods and verifying a significantly greater proportion of LLM-generated code. Background LLMs for Code Generation. Recent years have witnessed rapid progress in large language models (LLMs) ( Achiam et al. 2023 ; Touvron et al. 2023 ; Liu et al. 2024a ) , driven by advances in neural architectures and the availability of massive text datasets. These models, originally developed for natural language understanding and generation, have demonstrated remarkable generalization abilities across a wide range of tasks. With the inclusion of large-scale code corpora in their training data, LLMs have also shown strong performance in code-related domains ( Guo et al. 2024 ; Li et al. 2023b ; Chowdhery et al. 2023 ; Fried et al. 2022 ) . Leading commercial tools such as GitHub Copilot, OpenAI Codex, and Google AlphaCode exemplify the integration of LLMs into practical software engineering workflows. These tools can assist developers with code completion, code synthesis from natural language descriptions, automatic bug fixing, and code translation between languages. LLM-based coding assistants have been adopted across various domains, including web development, data science, education, and competitive programming. Their ability to handle diverse programming languages and understand both code and natural language makes them increasingly valuable in both industry and academia. Traditional Code Verification Pipelines. Despite the remarkable progress in code generation, ensuring the correctness and reliability of automatically generated code remains a core challenge. Traditional software engineering relies on well-established verification pipelines, typically including syntax checking ( Stefik and Siebert 2013 ) , compilation ( Fehnker et al. 2007 ) or interpretation, and functional testing ( Yusifoğlu, Amannejad, and Can 2015 ) . While these automated tools can efficiently detect surface-level errors, the correction of detected issues remains largely manual and experience-driven. Human developers are responsible for interpreting error messages, diagnosing root causes, and applying fixes—creating a significant bottleneck