RAG-Stack : Co-Optimizing RAG Quality and Performance From the Vector Database Perspective DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06
Wenqi Jiang · 2025
Read RAG-Stack : Co-Optimizing RAG Quality and Performance From the Vector Database Perspective DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Wenqi Jiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20296.txt.
Introduction
Wenqi Jiang presents RAG-Stack, a blueprint for jointly improving the quality and the speed of retrieval-augmented generation systems from the vector database point of view. Such systems fetch documents from a database and feed them into a large language model prompt, which improves reliability. Tuning them is hard because quality and performance depend on many knobs across models, databases, software, and hardware.
The blueprint has three pillars working together. An intermediate representation separates quality concerns from performance concerns in one shared description. A cost model estimates system performance for any such description. A plan exploration algorithm then searches those descriptions for configurations that are both high quality and high performance. Jiang argues this structure should become the standard approach.
Catalog record
| CREATOR | Wenqi Jiang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20296
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RAG-Stack : Co-Optimizing RAG Quality and Performance From the Vector Database Perspective DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Wenqi Jiang Abstract . Retrieval-augmented generation (RAG) has emerged as one of the most prominent applications of vector databases. By integrating documents retrieved from a database into the prompt of a large language model (LLM), RAG enables more reliable and informative content generation. While there has been extensive research on vector databases, many open research problems remain once they are considered in the wider context of end-to-end RAG pipelines. One practical yet challenging problem is how to jointly optimize both system performance and generation quality in RAG, which is significantly more complex than it appears due to the numerous knobs on both the algorithmic side (spanning models and databases) and the systems side (from software to hardware). In this paper, we present RAG-Stack , a three-pillar blueprint for quality–performance co-optimization in RAG systems. RAG-Stack comprises: (1) RAG-IR , an intermediate representation that serves as an abstraction layer to decouple quality and performance aspects; (2) RAG-CM , a cost model for estimating system performance given an RAG-IR ; and (3) RAG-PE , a plan exploration algorithm that searches for high-quality, high-performance RAG configurations. We believe this three-pillar blueprint will become the de facto paradigm for RAG quality–performance co-optimization in the years to come. 1. Introduction Vector database has become a hot topic in recent years due to the rapid advancement of machine learning and generative AI. Among its many applications, Retrieval-Augmented Generation (RAG) has emerged as one of the most prominent use cases in the LLM era. By incorporating relevant documents retrieved from databases into the LLM prompt as shown in Figure 1 , RAG improves the reliability, timeliness, and comprehensiveness of generated responses ( Borgeaud et al. 2022 ; Lewis et al. 2020 ; Edge et al. 2024 ) . As a result, RAG has become the industry standard for knowledge-intensive natural language processing tasks ( Borgeaud et al. 2022 ; Shao et al. 2024 ; Wang et al. 2023 ; Lazaridou et al. 2021 ; Lewis et al. 2020 ) . Well-Tuned Vector Databases Alone Are Not Enough for RAG Serving. We have witnessed extensive research on optimizing query processing in vector databases (referred to as retrievals throughout this paper for brevity), particularly in terms of quality (e.g., recall) and performance (e.g., QPS, latency), often on standard benchmarks with a predefined dataset including a fixed number of encoded vectors, a specified Top- K , and a target recall requirement. While research on retrieval itself is undoubtedly fundamental, many open problems remain when integrating vector databases into end-to-end RAG pipelines, because the previous assumptions (optimizing retrievals in isolation on well-defined benchmarks) no longer hold. On the other hand, there is a vast number of configurable knobs in RAG spanning both algorithmic and system aspects, all of which influence end-to-end RAG system performance (e.g., latency, throughput) and generation quality (e.g., accuracy, reliability), making it essential to move beyond tuning vector databases in isolation. Figure 1 . A simple RAG pipeline with models and a database. Algorithm Configuration Space. There are numerous algorithmic factors that influence the generation quality of a RAG pipeline. For example, how is the database configured? This includes decisions such as document chunking strategies, choice of embedding models, index configurations, and retrieval quality requirements such as recall. In addition to the database, should auxiliary models be used before and after retrieval? These may include a query rewriter that rephrases or decomposes the user’s original question, and a result reranker that re-scores the relevance of the retrieved Top- K documents. Once relevant content is retrieved, how many documents should be incorporated into the prompt? Many factors that are typically taken for granted in standard retrieval benchmarks become essential variables in this end-to-end context. Notably, the optimal configuration not only depends on these many factors, but also (a) varies across applications and coupled LLMs, and (b) evolves as new retrieval techniques and models emerge. System Design Space. There is also a wide range of system design choices beyond the database itself, as performance and efficiency must be optimized for the entire RAG pipeline as a whole. First, there are software-level options, such as the choice of databases and LLM serving frameworks. Second, there are various hardware options: depending on the type and availability of inference hardware, should the database run on CPUs (in-memory or on-disk), leverage accelerators such as GPUs or FPGAs, or combine different hardware in a heterogeneous setup? Third, end-to-end RAG scheduling and resource management decisions must be made, such as how to allocate hardware resources across RAG components (e.g., database, query rewriter, result reranker, and the main LLM) and how to configure batching policies. Just as optimal RAG algorithm configurations evolve, optimal system configurations for a given RAG algorithm will also shift over time as software and hardware continue to advance. Research Gap: Co-Optimizing Quality and Performance. Given the many additional dimensions introduced by RAG, the optimization objective naturally shifts from optimizing recall and QPS in standalone database to advancing the performance–quality Pareto frontier for end-to-end RAG serving. Due to the complexity of the RAG landscape as explained above, it is not surprising that existing research has focused on optimizing either the quality or performance. One might suggest that performance–quality co-optimization for RAG can be achieved through expert-based tuning, but such an approach is often suboptimal and, in many cases, impractical. This is not only because it is challenging to find an individual or a tightly coordinated team with expertise spanning RAG algorithms, database systems, and inference software/hardware. More importantly, as discussed earlier, the optimal RAG algorithm–system configuration varies across applications and evolves over time with the emergence of new retrieval techniques, models, and software/hardware stacks, making expert knowledge brittle and quickly outdated. Figure 2 . RAG-Stack systematically explores the performance–quality Pareto frontier for RAG serving. RAG-Stack Overview. In this paper, we propose RAG-Stack, a three-pillar blueprint for systematically co-optimizing quality and performance in RAG serving. As shown in Figure 2 , RAG-Stack consists of: (1) RAG-IR , an I ntermediate R epresentation and abstraction layer that bridges RAG algorithms and systems; (2) RAG-CM , a C ost M odel on the systems side that predicts performance given a RAG algorithm; and (3) RAG-PE , a P lan E xploration strategy that navigates the RAG configuration space. The key idea is to let RAG-PE iteratively explore the RAG algorithm space, where in each iteration, the generation quality is evaluated and the maximum achievable system performance is predicted by RAG-CM using the algorithm’s RAG-IR representation. Guided by prior evaluations of quality and performance, RAG-PE predicts the next configuration to explore that is likely to approach the performance–quality Pareto frontier. We now elaborate on each of these three pillars: Abstraction. First, RAG-IR provides a well-defined abstraction between the quality and performance dimensions, while remaining extensible to emerging RAG algorithm–system co-design techniques. The goal of this abstraction is to allow ML and systems researchers to consider the quality and performance optimization separately. At its core, RAG-IR is a collection of dataflow graphs, where each node