CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06
Yushi Sun, Lei Chen · 2026
Read CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Yushi Sun, Lei Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26176.txt.
Introduction
Sun and Chen argue that model question answering over knowledge graphs suffers because planners rebuild retrieval plans from scratch, like a database without a plan cache. That invites schema hallucinations and thin coverage. Their CacheRAG architecture caches past plans with a schema-agnostic natural-language interface, a varied two-layer cache index, and bounded subgraph expansion with strict complexity limits. Experiments show large gains over leading baselines, including about thirteen percent higher accuracy and about eighteen percent higher truthfulness on a hard retrieval benchmark.
Catalog record
| CREATOR | Yushi Sun, Lei Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82278 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26176
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Yushi Sun, Lei Chen Abstract . The integration of Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) has significantly advanced Knowledge Graph Question Answering (KGQA). However, existing LLM-driven KGQA systems act as stateless planners, generating retrieval plans in isolation without exploiting historical query patterns: analogous to a database system that optimizes every query from scratch without a plan cache. This fundamental design flaw leads to schema hallucinations and limited retrieval coverage. We propose CacheRAG, a systematic cache-augmented architecture for LLM-based KGQA that transforms stateless planners into continual learners. Unlike traditional database plan caching (which optimizes for frequency), CacheRAG introduces three novel design principles tailored for LLM contexts: (1) Schema-agnostic user interface: A two-stage semantic parsing framework via Intermediate Semantic Representation (ISR) enables non-expert users to interact purely in natural language, while a Backend Adapter grounds the LLM with local schema context to compile executable physical queries safely. (2) Diversity-optimized cache retrieval: A two-layer hierarchical index (Domain → \rightarrow Aspect) coupled with Maximal Marginal Relevance (MMR) maximizes structural variety in cached examples, effectively mitigating reasoning homogeneity. (3) Bounded heuristic expansion: Deterministic depth and breadth subgraph operators with strict complexity guarantees significantly enhance retrieval recall without risking unbounded API execution. Extensive experiments on multiple benchmarks demonstrate that CacheRAG significantly outperforms state-of-the-art baselines (e.g., +13.2% accuracy and +17.5% truthfulness on the CRAG dataset). 1. Introduction Knowledge Graph Question Answering (KGQA) is a fundamental task in data management and artificial intelligence, aiming to provide precise answers to natural language questions over structured knowledge bases. Recently, the integration of Large Language Models (LLMs) via the Retrieval-Augmented Generation (RAG) paradigm has emerged as a promising solution. By incorporating semantic parsing and query plan generation into the LLM’s reasoning process, these RAG-based systems can handle complex queries and generate context-aware responses. Figure 1. Comparison of stateless LLM execution (baseline) and CacheRAG (our approach) on a KGQA task. (a) Input: natural language question about J.K. Rowling’s latest magic novel. (b) Baseline suffers from schema hallucination (retrieving wrong entity The Hallmarked Man , retrieval of non-existent attributes such as topic ) and incomplete retrieval (missing the genre attribute). (c) CacheRAG workflow: the parser extracts ISR, semantic cache provides historical plans via plan retrieval, and bounded subgraph expansion dynamically adjusts the query plan. (d) Execution example: CacheRAG refers to the useful cached example, applies bounded depth/breadth expansion with dynamic adjustment to explore alternative schema paths ( genre , style , publishDate , character ), successfully identifying the correct KG subgraph for QA. However, when deployed in real-world scenarios, existing LLM-driven KGQA pipelines expose critical system-level flaws. Most prominently, these systems act as stateless planners : they generate retrieval plans (e.g., SPARQL queries or API calls) based solely on the current question, failing to leverage historical question-answering experiences . This design mirrors a hypothetical database system that optimizes every query from scratch without a plan cache: an inefficiency that modern DBMSs resolved decades ago through execution plan caching ( 8 ) . Yet, naively adapting traditional DB plan caching to LLM-based systems fails due to fundamental differences: (1) LLM context windows reward diversity over frequency (unlike LRU/LFU), and (2) retrieval plans must be schema-aligned to prevent hallucinations. The stateless KGQA reasoning approach inevitably leads to two major problems, as illustrated in Figure 1 (b): First, susceptibility to schema hallucination . Consider the query “Who are the characters in J. K. Rowling’s latest magic novel?” Existing stateless LLM planners might blindly retrieve her latest general novel ( The Hallmarked Man , a crime fiction) by missing the “magic” constraint. When forced to self-correct, they frequently hallucinate schema attributes, attempting to query a non-existent topic: magic predicate, leading to empty results and execution failures. Second, limited retrieval coverage . Even when the entity is correctly identified, zero-shot planning often generates shallow, incomplete query structures: missing multi-hop reasoning paths or failing to explore alternative schema branches when initial retrieval returns empty results. Root Cause and Our Solution. Root Cause 1: Schema Misalignment . Existing systems expect users to phrase queries using exact KG predicates (e.g., “novels with topic=magic”) instead of natural language. Our CacheRAG method addresses this through a two-stage decoupling architecture: (1) User Layer (Logical) : Users express intents naturally (“magic novel”). The LLM acts as a syntactic parser to extract an Intermediate Semantic Representation (ISR): {entity: J.K. Rowling, type: novel, constraints: [magic]}. (2) System Layer (Physical) : The Backend Adapter grounds the LLM by fetching the actual local schema (e.g., valid edges around the retrieved entity) and prompts the LLM to compile the logical ISR into an executable physical query. For instance, given the constraint “magic” and a local schema containing genre , the LLM safely maps the intent to the genre: fantasy predicate. This eliminates the need for user queries to match exact domain-specific predicate names while substantially mitigating schema hallucination. Root Cause 2: Lack of Historical Query Patterns . Traditional stateless planners reinvent query logic for every question, wasting API calls and producing shallow plans. As shown in Figure 1 (c), CacheRAG introduces a Semantic Cache that stores successful historical retrieval plans. During plan generation, the cache retrieval module provides useful examples (e.g., similar queries demonstrating “author \rightarrow novels \rightarrow genre” navigation patterns), enabling the LLM to learn complex multi-hop reasoning rather than relying on zero-shot guessing. When initial retrieval is insufficient, the heuristic dispatcher (Figure 1 (d)) triggers bounded depth/breadth expansion, exploring alternative schema paths (e.g., checking both genre and style predicates) until the correct answer is found. This dual-stage design ensures that non-expert users interact purely via natural language (addressing Root Cause 1), while the system leverages historical query patterns to guide plan generation (addressing Root Cause 2): eliminating the need for users to learn KG-specific predicates or SPARQL syntax (Section 3.1). Our Key Insight. Successful KGQA interactions exhibit strong structural reusability : for example, questions about movie awards consistently require similar multi-hop patterns (e.g., film \rightarrow director \rightarrow other films \rightarrow nominations), regardless of specific entities. Yet, stateless LLMs must rediscover these patterns for every query. CacheRAG bridges this gap by introducing a systematic cache-augmented architecture for LLM-based KGQA, transforming stateless planners into continual learners that exploit historical query patterns while maintaining rigorous schema fidelity and execution bounds. Designing such a robust cache-augmented system requires systematically addressing four fundamental challenges. For each challenge, CacheRAG introduces a targeted architectural design choice with strict justifications: • Challenge 1: Non-Deterministic Attribute Extraction. LLMs often hallucinate predicates (e.g., extracting non-existent attribute “topic”), leading to execution failures and making the system unintuitive for non-expert users. Solution 1: Schema-Constrained Semantic Parsing. We decouple the user-facing prompt from KG-specific jargon. The LLM first extracts an Intermediate Semantic Representation (ISR). A Backend Adapter then provides the valid local schema context, repurposing the LLM to compile the ISR into physical KG queries. This ensures schema fidelity and enables non-expert users to interact purely via natural language without learning SPARQL or KG schemas. • Challenge 2: Suboptimal Experience Replay (Cache Scheduling). Traditional cache metrics (LRU/LFU) or naive semantic similarity fall short