TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci, [0.4em] University of Washington · 2026
Read TraceLab: Characterizing Coding Agent Workloads for LLM Serving by Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci, [0.4em] University of Washington (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30560.txt.
Introduction
Kan Zhu and six colleagues release TraceLab, a record of how programmers actually use coding agents day to day. Serving these agents efficiently is hard, yet the workload data needed to improve serving barely exists. Public traces miss real usage across several agents and model families, so the team logged its own work with Claude Code and Codex.
The trace holds roughly forty-three hundred sessions, about three hundred fifty thousand model steps and four hundred thirty thousand tool calls. Analysis shows long autonomous loops and long contexts paired with short outputs, and tool calls with heavy-tailed variety, meaning a few types dominate while many are rare. Prefix caching, reusing earlier computation, hit often but imperfectly. The authors point to cheaper tool calling, smarter prefill planning, predicting tool delays, and better cache handling around pauses while humans think.
Catalog record
| CREATOR | Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci, [0.4em] University of Washington |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30560
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TraceLab: Characterizing Coding Agent Workloads for LLM Serving Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci, [0.4em] University of Washington Abstract Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day-to-day coding-agent usage across multiple agents and model families for serving-system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding-agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day-to-day use of Claude Code and Codex. Our analysis shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw-syfi/TraceLab.git ; the project website is https://tracelab.cs.washington.edu . \NoHyper * * footnotetext: Work done while interning at the University of Washington. \endNoHyper 1 Introduction Agentic LLMs have grown rapidly, enabling models to handle increasingly complex tasks through reasoning and tool calling [ 91 , 65 ] . Among these, coding agents are evolving especially fast, as companies release their own coding agents, e.g., Claude Code from Anthropic [ 4 ] , Codex from OpenAI [ 55 ] , and Gemini CLI from Google [ 19 ] . Open-source solutions such as OpenCode [ 58 ] and DeepCode [ 22 ] , which allow users to run local model deployments, are also gaining popularity. From a systems perspective, however, serving these coding agents is challenging. Contexts grow over sessions; SLOs are tight; and repeated tool calling places pressure on both the tool-serving infrastructure and the underlying LLM-serving engines. Understanding the characteristics of these workloads is a prerequisite for serving them efficiently. Existing benchmarks and traces only partially address this. Traces obtained from serving systems like Mooncake [ 63 ] and Splitwise [ 61 ] capture real user interactions, but they are not focused on coding agents and lack multi-step tool-call behavior. Capability benchmarks such as Terminal-Bench [ 51 ] and SWE-bench [ 30 ] measure whether an agent can solve a task. They contain relatively few tasks, each narrowly scoped to a single problem. While they are standard for evaluating model accuracy, they are not designed to capture the diversity of real-world coding-agent usage from a serving system’s perspective. We aim to help fill this gap. Luckily, coding agents will by default log conversations and tool calls, which serves as a rich source of data for understanding real-world usage. We build a pipeline that extracts the data, normalizes the format, anonymizes it to protect user privacy, and finally runs the analysis to understand the workload characteristics. In this paper, we share initial insights from what is, to our knowledge, the first large-scale cross-provider trace of real coding-agent usage: about 4,300 sessions comprising roughly 350,000 LLM steps and 430,000 tool calls, collected from 43 developers over roughly eight months and spanning both Claude Code and Codex across more than 20 model versions. From our collected traces, we analyze the implications for LLM serving systems. At the session level, the workload is largely autonomous and consists of multiple LLM invocations and tool calls. . To complete a user’s task, the agent will, on average, carry out 8.8 LLM calls and invoke tools 10.8 times. It takes on average 4.3 minutes to complete one request, with long tails whose p90 exceeds 6.4 minutes. For the majority of iterations, the context grows as more user input, tool results, and LLM generations are added. However, two categories of context reduction are observed: a compaction near the context limit and a micro context reduction unique to Codex that occasionally occurs when the user starts their next request. Due to the multi-step nature of sessions, to avoid repeatedly prefilling the existing conversation history, modern LLM serving systems widely adopt prefix caching [ 103 ] . Prefix caching preserves the KV cache for history tokens, so each step only needs to prefill newly appended tokens. Although in API pricing, reading from the prefix cache is roughly 10\times cheaper than performing a fresh prefill [ 5 , 56 , 57 ] , the accumulated history in one session can become long, and the cache must be read at every step. As a result, we observe that prefix-cache reads dominate the overall API cost, under the pricing snapshot in table 6 . Within a step, LLM generation features long context but short output. Despite our expectation of long generations due to reasoning, the frequently occurring tool calls cut LLM generation into multiple steps, making each step’s LLM invocation have shorter output length than traditional reasoning workloads. The median LLM generation workload is about 119K prefix tokens (prefix cache read of history context), 875 append tokens (fresh new input tokens from user or tool results), and 214 output tokens. Despite the long context, both providers achieve good decode speed. The normalized decode speed has a median of 46.8 normalized tokens per second for Claude and 33.9 for Codex, both with significant variance (CV > 50%). For Codex, the median pure decode speed is 61.3 with an estimated TTFT of 3.1 seconds per step. Additionally, tool calls are diverse but heavily tailed for popularity and latency. While more than 80 distinct tools are observed, the distribution is heavily skewed. The top 3 tools—Bash, Read, and Edit for Claude; exec_command, write_stdin, and apply_patch for Codex—account for more than 80% of all tool calls. The latency of tool calls is also diverse, spanning from milliseconds to hours. Tool calls longer than 1 min are only 4 percent of all tool calls, but they account for 85% of total tool-call time. Finally, the prefix cache greatly reduces the cost, but misses are still expensive . The global prefix cache hit rate is 95.7%, while most misses occur between the last request’s final output and the next request’s user input, as the delay due to human reading, thinking, and typing can frequently exceed the prefix cache eviction time. Overall, cache misses cause 3.8 \times more tokens to be prefilled than prefills due to truly unique input tokens. Given these insights, we highlight the following research directions on optimizing LLM serving for coding agents: 1. The overhead of frequent tool-LLM switch motivates denser or fused tool invocations and lower-overhead tool call approval/runtime paths. 2. The mixture of short and long append lengths motivates append-length-aware prefill routing, along with adaptive selection of kernels and serving-engine parallelism. 3. The large latency variation for each tool type suggests that tool-latency prediction for KV cache eviction policy should consider the semantics of the requested operation and recent latency history, not just the tool name. 4. The long prefix motivates sparse attention for reducing the decoding cost. 5. The frequent idle gaps necessitate cheaper KV cache storage, a better compression algorithm, and wise KV-cache eviction, or prefetching policies around those