Runtime-Structured Task Decomposition for Agentic Coding Systems Conference: ACM Workshop on Agentic Software Engineering (ACM CAIS 2026); May 26–29, 2026; San Jose, CA CCS: Computing methodologies Artificial intelligence CCS: Computer systems organization Architectures
Shubhi Asthana, Bing Zhang, Chad DeLuca, Hima Patel, Ruchi Mahindru · 2026
Read Runtime-Structured Task Decomposition for Agentic Coding Systems Conference: ACM Workshop on Agentic Software Engineering (ACM CAIS 2026); May 26–29, 2026; San Jose, CA CCS: Computing methodologies Artificial intelligence CCS: Computer systems organization Architectures by Shubhi Asthana, Bing Zhang, Chad DeLuca, Hima Patel, Ruchi Mahindru (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15425.txt.
Introduction
Shubhi Asthana and four colleagues propose runtime-structured task decomposition for agentic coding systems, replacing monolithic prompts with executable control logic. Monolithic designs prove brittle and costly, since failures force full reruns. Static decomposition does not always help: on a Kubernetes root-cause workload it cost more tokens than the monolith. In the runtime-structured design, language models handle only focused judgments, and outputs validate against schemas before downstream use. Evaluations across two workloads and three configurations show the structured approach cutting retry costs.
Catalog record
| CREATOR | Shubhi Asthana, Bing Zhang, Chad DeLuca, Hima Patel, Ruchi Mahindru |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18095 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15425
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Runtime-Structured Task Decomposition for Agentic Coding Systems Conference: ACM Workshop on Agentic Software Engineering (ACM CAIS 2026); May 26–29, 2026; San Jose, CA CCS: Computing methodologies Artificial intelligence CCS: Computer systems organization Architectures Shubhi Asthana, Bing Zhang, Chad DeLuca, Hima Patel, Ruchi Mahindru Abstract . Agentic coding systems increasingly deploy LLMs for multi-step software engineering tasks—debugging, root cause analysis, code review—yet most encode all task logic, control flow, and output generation within monolithic prompts. This leads to brittle behavior, poor debuggability, and expensive full-pipeline reruns when any step fails. We present runtime-structured task decomposition , an architectural pattern in which task partitioning decisions are governed by executable control flow rather than static prompt text, and LLMs are invoked only for narrowly scoped judgment tasks with schema-validated outputs. We evaluate this pattern on two software engineering workloads across three configurations —monolithic, static decomposition (same subtask graph, no runtime branching), and runtime-structured—over 10 runs each. Across both workloads, we find that decomposition structure alone does not reliably reduce retry cost. In the Kubernetes root cause analysis workload, the static baseline’s retry cost ( 1,632 ± 145 1{,}632\pm 145 tokens) exceeds the monolithic baseline ( 904 ± 17 904\pm 17 tokens) by 80.5%, because fixed sequential execution must rerun multiple downstream subtasks. In the multi-file debugging workload, the same effect appears at smaller magnitude (933 vs. 703 tokens). Runtime-structured decomposition, by retrying only the failed subtask, reduces retry cost in both settings—to 436 ± 132 436\pm 132 tokens (RCA) and 460 tokens (debugging)—achieving up to a 51.7 % 51.7\% reduction over monolithic and a 73.2 % 73.2\% reduction over static decomposition. We discuss implications for agentic coding system design, production debugging, and AgentOps workflows. 1. Introduction Agentic coding systems—such as those built on AutoGen, LangGraph, and Claude Code—are among the earliest LLM agents deployed in real software engineering workflows. When a debugging agent fails, does it retry the full analysis or just the broken step? When a root cause analysis pipeline produces a malformed output, can it recover without re-ingesting thousands of log lines? These are production engineering questions with architectural answers. Most existing agentic coding systems encode task logic, control flow, and output generation within monolithic prompts, leading to brittle behavior, limited debuggability, and expensive full-pipeline reruns when failures occur. Recent frameworks such as DSPy ( 4 ) , LangGraph ( 6 ) , and AutoGen ( 11 ) have introduced programmatic abstractions for orchestrating LLM calls. However, task decomposition strategies in these frameworks are typically specified statically by the developer—the structure of the task must be known and fixed at design time. When intermediate outputs are invalid or branching conditions emerge at runtime, a static decomposition cannot adapt. In this paper, we present runtime-structured task decomposition (RSTD) , an architectural pattern in which task partitioning decisions are governed by executable control flow and validated intermediate signals rather than static prompt text. Under this paradigm: (1) LLMs are invoked only for narrowly scoped judgment tasks with schema-validated outputs. (2) Orchestration, branching, and state management are handled deterministically in code. (3) Failures are isolated to individual subtasks, enabling selective retry without full pipeline rerun. We implement this pattern using the Mellea generative computing framework ( 8 ) and evaluate it across three configurations —monolithic, static decomposition (same subtask structure, no runtime branching), and runtime-structured—over 10 runs each. The three-way comparison isolates the contribution of runtime branching from decomposition structure alone. A key empirical finding: static decomposition alone does not reliably reduce retry cost relative to monolithic, and can increase it when failures trigger cascading re-execution of downstream subtasks. Its retry cost ( 1{,}632\pm 145 tokens) exceeds the monolithic baseline ( 904\pm 17 tokens) by 80.5%, because fixed sequential execution must rerun all downstream subtasks from the point of failure. Runtime-structured decomposition, by retrying only the failed subtask ( 436\pm 132 tokens), achieves a 51.7\% retry cost reduction over monolithic and a 73.2\% reduction over static decomposition—demonstrating that runtime branching, rather than decomposition structure alone, is the primary mechanism. Our contributions are: (1) A formalization of RSTD as an architectural pattern for agentic coding systems, distinct from both monolithic prompting and static decomposition in DSPy, LangGraph, and AutoGen. (2) A three-configuration empirical evaluation across 10 runs per setting, with variance quantified, framework overhead separated from LLM API latency, and a static baseline that isolates the contribution of runtime branching. (3) Operational implications for agentic coding systems: subtask-level monitoring, selective retry under failure, and per-subtask model substitution. 2. Architecture Runtime-structured task decomposition separates orchestration, state management, and LLM inference into distinct layers. Rather than embedding task structure inside prompts, decomposition decisions are externalized into executable control flow, enabling task structure to be determined dynamically at runtime. The system consists of three components, illustrated in Figure 1 : Figure 1. Runtime-structured decomposition architecture. A decomposition engine governs control flow, invoking typed LLM judgment operators over bounded context. Outputs are validated and stored in persistent state, feeding back into subsequent decomposition decisions. • Decomposition Engine : Governs runtime task partitioning via developer-authored conditional logic—branching on schema validation outcomes, confidence thresholds, or predecessor completion status. Branching decisions are resolved at runtime against validated intermediate state. • Judgment Operators : Typed LLM calls scoped to a single reasoning task. Each call declares an explicit output schema; validation failure triggers a targeted repair prompt rather than full pipeline rerun. • State Manager : Persists validated intermediate results keyed by subtask identifier. Downstream subtasks access prior state explicitly by key, keeping per-call context bounded. Branching decisions evaluate three runtime signals: (1) schema validation outcome; (2) output content (non-empty result or confidence above threshold); and (3) subtask completion status. All policies are deterministic conditional logic, making them auditable, unit-testable, and reproducible. Mono. Monolithic Prompt (all logic) Fail Full Rerun Static S1 S2 S3 S4 Fail reruns S2 S3 S4 RSTD S1 S2 S3 S4 Fail Retry S3 only Figure 2. Retry behavior under subtask failure. Monolithic reruns the full pipeline; static reruns all downstream subtasks; RSTD retries only the failed subtask. We operationalize this architecture using the open-source Mellea framework ( 8 ) , which provides typed LLM calls with enforced input/output schemas, persistent session state, and automated validation-and-repair loops. A Mellea call specifies a prompt and a set of output requirements: result = session.instruct( prompt = "Classify anomalies. Output JSON: " "[{anomaly, type, severity, conf}]", requirements = [ "Output must be a JSON list.", "Each item must have: anomaly, type, severity, confidence." ] ) # On failure, Mellea retries with the # validation error appended. 3. Case Studies We demonstrate RSTD through two software engineering workloads evaluated across three configurations : (i) Monolithic —all task logic in a single prompt; (ii) Static —same subtask graph, fixed sequential execution, no runtime branching; and (iii) RSTD —our approach, with runtime-controlled branching and selective retry. The static baseline isolates whether retry advantages come from decomposition structure alone or specifically from runtime branching. All agents use gpt-4 at temperature 0. Token counts are measured using tiktoken . Each configuration is run 10 times ; we report mean \pm standard deviation. Non-zero standard deviations at temperature 0 reflect API-side variability (e.g., request scheduling and batch-size variation) rather than sampling randomness. Monolithic baselines are drawn from https://github.com/microsoft/autogen