Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines
Alimurtaza Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui · 2026
Read Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines by Alimurtaza Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20630.txt.
Introduction
Alimurtaza Merchant and five colleagues speed up industrial agent pipelines for asset operations, where one query can coordinate sensors, work orders, and forecasting tools. Standard caching breaks down when answer validity depends on time, asset, or sensor values. They add a temporal semantic cache plus workflow optimizations, including disk-backed tool discovery and dependency-aware parallel steps. Workflow changes alone gave a 1.67-fold speedup, while cache hits reached a median 30.6-fold speedup.
Catalog record
| CREATOR | Alimurtaza Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37275 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20630
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines Alimurtaza Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui Abstract Industrial asset operations workflows are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure modes, forecasting tools, and domain-specific agents. We evaluate this problem on AssetOpsBench (AOB), an industrial agent benchmark whose plan-execute pipeline exposes repeated overhead from tool discovery, LLM planning, MCP tool execution, and final summarization. Existing LLM caching techniques such as KV-cache reuse and embedding-based semantic caching were designed for chatbot serving and break down when output validity depends on time, asset, or sensor parameters. We propose two complementary optimization layers for AOB plan-execute pipelines: a temporal semantic cache and a set of MCP workflow optimizations combining disk-backed tool-discovery caching and dependency-aware parallel step execution. MCP workflow optimizations corresponded to a 1.67× speedup and reduced median end-to-end latency by about 40.0% while the temporal-cache benchmark achieved a median of 30.6× speedup on cache hits. Beyond the speedup, our results expose a concrete failure mode of pure semantic caching for parameter-rich industrial queries, providing a critical analysis of how caching choices interact with evaluation correctness in MCP-backed agent benchmarks. 1 Introduction LLM-based agents increasingly serve as orchestration layers over domain-specific tools and data sources ( 27 ; 20 ; 17 ) . Many such agents follow the Plan-Execute paradigm: a planner LLM decomposes a user query into a sequence of tool calls, and an executor invokes those tools, often through standardized interfaces such as the Model Context Protocol (MCP) ( 1 ) . While expressive, this two-stage structure introduces substantial wall-clock latency. A single query can require tool discovery, multi-step planning, multiple MCP tool invocations, and a final summarization pass before an answer is returned. This latency is particularly acute in industrial asset operations, where queries naturally span heterogeneous data sources: sensor telemetry, work orders, failure modes, and time-series forecasts. The AssetOpsBench benchmark ( 16 ) formalizes this setting with four MCP-backed domain servers and a corpus of human-authored operational queries. In practice, the same operator may issue many semantically related queries against the same assets: paraphrases, repetitions, parameter shifts (Chiller 6 vs. Chiller 9), or time-window shifts (yesterday vs. last week). A naive plan-execute implementation pays the full orchestration cost on every query, and at paraphrase scale this makes systematic evaluation of MCP-backed agents prohibitively slow. Caching is the standard remedy for repeated computation in LLM serving, but existing techniques were designed for chatbot workloads. Context (KV) caching ( 6 ; 26 ; 8 ; 10 ) reuses prefill states for identical prefixes; semantic caching ( 2 ; 21 ) reuses (input, output) pairs across paraphrases via embedding similarity. Neither approach matches the structure of industrial agent queries, where output validity depends on external state (asset, sensor, time window) that is not visible in the query embedding. Recent work on Agentic Plan Caching ( 28 ) addresses part of this gap by caching plan templates rather than answers, but does not address temporal validity, which is central in industrial telemetry settings where “what happened yesterday” resolves to a different window each day. Our approach. We propose two optimization layers for AOB plan-execute pipelines. At the query level, we build a temporal semantic cache with a lightweight temporal classifier that routes each query into one of four buckets: Volatile (live state, bypass cache), Static (no temporal dependence, standard semantic match), Relative (e.g., “yesterday,” resolved into a concrete window), or Anchored (fixed time window, matched against compatible windows). Static and (resolved) Anchored queries enter embedding-based retrieval followed by a reranker-based judge. At the workflow level, we add two MCP optimizations : disk-backed tool-discovery caching and dependency-aware parallel step execution over a persistent server pool. The two layers are independently beneficial and additive: the MCP layer reduces latency on every query regardless of cache state, while the cache layer adds large further savings when a query resolves to a valid hit. Contributions. 1. Temporal semantic caching for industrial agents. We extend semantic caching with a pre-retrieval temporal classifier and a window-aware judger that addresses the parameter-and-time sensitivity of industrial queries. 2. MCP workflow optimizations. We combine discovery-phase caching, DAG-layered parallel execution, and a persistent server pool to reduce per-query orchestration overhead in MCP-backed plan-execute pipelines. 3. Refined evaluation setup for AOB. We provide a paired baseline-vs-optimized harness, a paraphrase-tier test set with parent-id-based ground truth for cache hit/miss labelling, and per-phase latency profiling that makes systematic AOB ablations tractable on a single machine. 4. Critical analysis of caching as an evaluation choice. We expose a concrete failure mode: pure semantic similarity is not a sound proxy for answer validity in parameter-rich AOB queries, capping hit-decision F1 near 0.67 in our setting. This gives a measurable handle on when caching is safe to use as part of an evaluation pipeline, and when it is not. 2 Background and Motivation 2.1 AssetOpsBench and MCP-backed Plan-Execute Agents AssetOpsBench (AOB) ( 16 ) is a benchmark for evaluating LLM agents on industrial asset operations and maintenance workflows. AOB exposes four specialized domain servers covering Internet of Things (IoT) telemetry, Failure Mode and Sensor Relation (FMSR), Time Series Foundation Models (TSFM) ( 4 ) , and Work Order (WO) records, all wrapped under the Model Context Protocol (MCP) ( 1 ) . The benchmark scenarios are written as human-authored natural-language operational queries rather than database-level API calls, reflecting the language an operator or reliability engineer would actually use. In a Plan-Execute pipeline ( 27 ) , a query is not answered by a single LLM call. Instead, the workflow decomposes into four phases: Discovery , which spawns MCP servers and collects tool signatures via list_tools() ; Planning , which uses an LLM to convert the query and tool catalog into a structured plan; Execution , which resolves tool arguments and invokes the relevant MCP tools; and Summarization , which uses an LLM to synthesize the tool outputs into a response. Figure 1 illustrates this structure and illustrates the baseline path. Figure 1 : MCP Workflow. The Plan-Execute abstraction is useful because it exposes a structured plan before tool execution begins. However, this separation does not automatically imply parallelism: many implementations consume the generated plan strictly sequentially. The optimization opportunity comes from treating the plan as a directed acyclic graph and dispatching dependency-independent steps concurrently, while preserving order across true dependencies. 2.2 LLM Caching for Agents: Methods and Limitations Caching is one of the most widely adopted techniques for reducing LLM serving cost. Context caching ( 6 ; 26 ; 8 ; 10 ) stores key-value pairs from the prefill phase and reuses them when prompt prefixes recur. Semantic caching ( 2 ; 21 ) stores (input, output) pairs and matches new queries by embedding similarity, exploiting the fact that paraphrases share underlying intent. Plan caching ( 28 ) caches plan templates extracted from completed agent executions and adapts them to new queries with a lightweight model. We find that all three families have limitations specific to MCP-backed industrial agent benchmarks like AOB: 1) Static-Output Assumption. Semantic caching assumes that outputs depend only on the input prompt ( 2 ; 21 ) . This