In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks
Simon Dennis, Michael Diamond, Rivaan Patil, Kevin Shabahang, Hao Guo · 2026
Read In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks by Simon Dennis, Michael Diamond, Rivaan Patil, Kevin Shabahang, Hao Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27891.txt.
Introduction
Dennis and four colleagues ask whether packing a full procedure into the system prompt can beat outside agent orchestration on set tasks. They run 200 chats per setup across travel booking, Zoom support, and insurance claims, judged on five quality checks. The in-context route fails far less: travel fails hit 24 percent under orchestration versus 11.5 percent in context, Zoom 9 percent versus under 1 percent, and insurance 17 percent versus 5 percent. The authors argue that newer models no longer need an outside orchestrator for chats that follow a set procedure.
Catalog record
| CREATOR | Simon Dennis, Michael Diamond, Rivaan Patil, Kevin Shabahang, Hao Guo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62274 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27891
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks Simon Dennis, Michael Diamond, Rivaan Patil, Kevin Shabahang, Hao Guo Abstract Agent orchestration frameworks—LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, and others—place an external orchestrator above the LLM, tracking state and injecting routing instructions at every turn. We present a controlled comparison showing that for procedural tasks, this architecture is dominated by a simpler alternative: putting the entire procedure in the system prompt and letting the model self-orchestrate. Across three domains—travel booking (14 nodes), Zoom technical support (14 nodes), and insurance claims processing (55 nodes)—we evaluate 200 conversations per condition using LLM-as-judge scoring on five quality criteria. The in-context approach scores 4.53–5.00 on a 5-point scale while a LangGraph orchestrator using the same model scores 4.17–4.84. The orchestrated system fails on 24% of travel, 9% of Zoom, and 17% of insurance conversations, compared to 11.5%, 0.5%, and 5% for the in-context baseline. While external orchestration may have been necessary for earlier models, advances in frontier model capabilities have made it unnecessary for multi-turn conversations following a defined procedure. 1 Introduction Substantial engineering effort has gone into building agent orchestration frameworks. LangGraph, CrewAI, Google ADK, OpenAI’s Agents SDK, Microsoft’s Semantic Kernel, Amazon’s Strands, and LlamaIndex all implement the same pattern: an external orchestrator sits above the LLM, tracking conversation state and injecting routing instructions at every turn. These frameworks represent significant investment—both by the companies building them and by the developers adopting them. Wang et al. 2025 found that 80% of developers struggle to select among these frameworks, with LangChain’s abstractions requiring “traversing seven layers of code” for a single change. Meanwhile, the reliability costs of orchestration are well documented: cascading errors, routing failures, and poor cross-trial consistency ( Cemri et al. 2025 ; Zhu et al. 2025 ; Gupta 2026 ; Kapoor et al. 2024 ) . This paper presents a simple empirical finding: for procedural tasks—conversations that follow a defined workflow from intake to resolution—the orchestration architecture is not just unnecessary but actively harmful. A frontier model given the entire procedure in its system prompt and left to self-orchestrate consistently outperforms the same model when orchestrated via LangGraph, across every domain and every metric we tested. The bottleneck is not better orchestration; orchestration itself is the bottleneck. Our contributions are: • A controlled comparison between a LangGraph orchestrator and an in-context approach using the same model (Claude Sonnet) across three domains ( n=200 per condition per domain), showing that in-context prompting dominates orchestration on all five quality criteria in all three domains. • An efficiency analysis showing that orchestration requires 1.2–1.7 \times more LLM calls per conversation due to routing overhead, while achieving lower quality. • A failure mode analysis revealing that the orchestrated system fails on 9–24% of conversations across domains despite using a frontier model, compared to 0.5–11.5% for the in-context baseline—orchestration introduces failure modes that would not otherwise exist. 2 Method The experiment compares simulated customer-service conversations produced under two conditions: (i) an agent orchestrated by LangGraph according to a given procedural workflow represented by a directed graph; (ii) an agent given the same workflow graph in context and otherwise allowed to self-orchestrate. In both conditions, the agent begins with the same initial problem scenario (see Appendices B – D for examples), and the conversation unfolds through dynamic interaction with a simulated user powered by Claude Sonnet. The user simulator receives the same scenario specification in both conditions but generates responses based on the evolving conversation history; it has no knowledge of the underlying flowchart and simply role-plays a customer with given preferences (Appendix H.2 ). We evaluate n=200 scenarios per condition per domain (1,200 total conversations), with scenarios designed to cover the full range of flowchart paths, user styles (specific to vague), satisfaction levels (enthusiastic to skeptical), and domain-specific variation. The resulting conversations are then evaluated by an LLM judge across several quality metrics, as described below. 2.1 Procedure Definition We represent procedures as directed graphs F=(N,E,n_{0},T,D) : • N : Nodes with role (agent/user) and prompt template • E\subseteq N\times N\times C : Edges with optional conditions • n_{0}\in N : Start node • T\subseteq N : Terminal nodes (success, abandonment, escalation) • D\subseteq N : Decision hubs—agent nodes with multiple outgoing edges whose conditions partition the conversation state (e.g., “information complete” vs. “missing documents”) We evaluate on three domains spanning different types and scales of complexity (flowcharts in Appendix E ): Travel booking (14 nodes, 3 decision hubs, 3 terminal states). Decision hubs route among 4–6 alternatives based on information completeness, user satisfaction with options, and booking confirmation status. The flowchart includes cycles (e.g., assessment \to gathering \to re-assessment). Zoom technical support (14 nodes, 3 decision hubs, 3 terminal states). A product-specific domain: the agent is a Zoom specialist with knowledge of Zoom’s UI, settings, and error codes. Triage routes based on initial diagnostics, retry logic handles troubleshooting failures, and final routing confirms resolution. Unlike travel booking, this domain requires accurate technical guidance—not just conversational flow. Insurance claims processing (55 nodes, 6 decision hubs, 4 terminal states). This domain tests scaling to additional complexity—nearly 4 \times the node count of the other domains. The procedure spans five phases: intake and claim type assessment, information gathering with per-type branching, coverage determination with exclusion analysis, settlement negotiation with offer/counter-offer loops, and resolution with follow-up tracking. The 6 multi-way decision hubs yield hundreds of unique paths through nested loops and cross-phase dependencies. 2.2 Two Conditions We compare two approaches using the same frontier model (Claude Sonnet 4.5 1 1 1 claude-sonnet-4-5-20250929 ) on the same scenarios: 1. LangGraph orchestrated : We implement the orchestration condition using LangGraph ( LangChain, Inc. 2024 ) , the most widely adopted agent orchestration framework. Each flowchart node maps to a LangGraph graph node. When the graph execution reaches an agent node, that node’s prompt template is inserted into the system prompt before calling the LLM. At decision hubs, the orchestrator makes an additional LLM call to select which edge to follow based on the conversation history and available conditions. User nodes invoke the user simulator. State flows through a dictionary tracking conversation history, current node, and metrics. 2. In-context baseline : The model receives the entire serialized flowchart in its system prompt—all nodes, edges, conditions, and terminal states rendered as structured text—and self-orchestrates. No external state tracking, no routing calls. One API call per agent turn. Agent system prompts are in Appendix H ; the per-node prompt templates injected by the orchestrator are in Appendix F . This design isolates the effect of orchestration architecture. Both conditions use the same model, the same procedure, and the same evaluation scenarios. The only difference is whether the model is orchestrated externally via LangGraph or manages the procedure internally. Appendices B – D show side-by-side conversation excerpts from both conditions on identical scenarios. The failure modes we observe—fragmented reasoning, routing errors at decision hubs, and constrained naturalness—are structural consequences of the orchestration architecture: we expect similar properties in any system that injects per-node instructions and routes via additional LLM calls. 2.3 LLM-as-Judge Evaluation We use an approach-agnostic LLM-as-judge methodology ( Zheng et al. 2023 ) where Claude Sonnet evaluates conversation quality without knowing which system