SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answering CCS: Information systems Information retrieval CCS: Applied computing Service-oriented architectures CCS: Computing methodologies Distributed artificial intelligence
Michael Iannelli, Sneha Kuchipudi, Vera Dvorak · 2024
Read SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answering CCS: Information systems Information retrieval CCS: Applied computing Service-oriented architectures CCS: Computing methodologies Distributed artificial intelligence by Michael Iannelli, Sneha Kuchipudi, Vera Dvorak (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06832.txt.
Introduction
Michael Iannelli, Sneha Kuchipudi, and Vera Dvorak tune multi-agent answer systems to meet service deals in a 2024 arXiv paper. Search-aided generation lets language models use fresh facts. Firms stack agents in rows or spread them wide to lift scores. Real jobs, though, carry deals on cost, speed, and quality that pull in mixed ways.
Their method maps deal goals to system knobs, then reshapes the agent crew on the fly. A question case study shows the crew trade quality for cost as deals shift. Query type and live load steer the reshaping. The frame suits real answer posts with tight deals.
Catalog record
| CREATOR | Michael Iannelli, Sneha Kuchipudi, Vera Dvorak |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-06832
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answering CCS: Information systems Information retrieval CCS: Applied computing Service-oriented architectures CCS: Computing methodologies Distributed artificial intelligence Michael Iannelli, Sneha Kuchipudi, Vera Dvorak Abstract . Retrieval Augmented Generation (RAG) enables Large Language Models (LLMs) to generalize to new information by decoupling reasoning capabilities from static knowledge bases. Traditional RAG enhancements have explored vertical scaling—assigning subtasks to specialized modules—and horizontal scaling—replicating tasks across multiple agents—to improve performance. However, real-world applications impose diverse Service Level Agreements (SLAs) and Quality of Service (QoS) requirements, involving trade-offs among objectives such as reducing cost, ensuring answer quality, and adhering to specific operational constraints. In this work, we present a systems-oriented approach to multi-agent RAG tailored for real-world Question Answering (QA) applications. By integrating task-specific non-functional requirements — such as answer quality, cost, and latency — into the system, we enable dynamic reconfiguration to meet diverse SLAs. Our method maps these Service Level Objectives (SLOs) to system-level parameters, allowing the generation of optimal results within specified resource constraints. We conduct a case study in the QA domain, demonstrating how dynamic re-orchestration of a multi-agent RAG system can effectively manage the trade-off between answer quality and cost. By adjusting the system based on query intent and operational conditions, we systematically balance performance and resource utilization. This approach allows the system to meet SLOs for various query types, showcasing its practicality for real-world applications. 1. Introduction Question Answering (QA) has been a pivotal area of research since the 1960s, aiming to provide precise answers to natural language questions. Unlike traditional search engines that return a list of hyperlinks to relevant pages, QA systems deliver direct answers to user queries ( Zhu et al. 2021 ) . The advent of generative Large Language Models (LLMs) has led search engines to integrate abstractive QA into their offerings—systems that generate answers which may not exist verbatim in source documents ( Danny D’Cruze 2024 ) . Additionally, new platforms specializing in abstractive QA, such as Perplexity ( Perplexity AI 2024 ) and ChatGPT Search ( OpenAI 2024 ) , have emerged. However, LLM-based abstractive QA exhibits notable limitations, including a lack of generalization to unseen data and a propensity for hallucination, i.e. the generation of false assertions ( Ji et al. 2023 ) . Retrieval Augmented Generation (RAG) ( Gao et al. 2024a ) addresses these shortcomings by combining LLMs with external knowledge sources, enabling the LLM to function as a reasoning layer rather than a static repository of facts. RAG retrieves context from external sources such as search engines, traditional databases, vector databases ( Jing et al. 2024 ) , or knowledge graphs ( Martino et al. 2023 ; Xu et al. 2024 ) , and utilizes the LLM for reasoning and response generation. Enhancements to RAG-based QA have primarily focused on decomposing the task into smaller subtasks and distributing responsibilities among specialized components. While these techniques have improved answer quality, they often introduce additional costs in terms of computation or increased response times. Real-world QA systems must navigate trade-offs between answer quality and the costs associated with providing those answers. Moreover, they must be dynamically reconfigurable to handle diverse Service Level Agreements (SLAs) and Quality of Service (QoS) requirements arising from various query intents and problem domains. For instance: • E-commerce Search: Highly sensitive to response times, where delays can significantly impact user experience and sales conversions ( van Vessum nd ; Achary 2023 ) . • Customer Support: Requires not just correct answers but stylistically compliant ones that adhere to the brand-voice guidelines of the domain or company ( Murár et al. 2024 ) . • Legal and Healthcare Queries: Demand high answer quality due to the critical nature of the information, with less emphasis on cost or latency constraints ( Samat 2024 ) . Furthermore, because modern cloud providers (e.g., SaaS and PaaS platforms) host these varied workloads side by side on shared infrastructure, the system’s overall utility is increased: by optimally provisioning and multiplexing resources across collocated services with diverse SLA requirements, we achieve higher utilization and more efficient service delivery. In addition to these domain-specific requirements, real-world RAG systems operate under constantly changing and sometimes adverse conditions. Factors such as network congestion, computational capacity constraints, and security concerns necessitate adaptive strategies. For example, during periods of high network traffic, minimizing external API calls and leveraging local LLM instances can help maintain answer quality while adhering to operational constraints. By incorporating knowledge about the task domain, QoS requirements, and the operational environment, we can quantify and manage the trade-offs between cost, answer quality, and other non-functional requirements. This enables the design of QA systems that are both efficient and effective across a range of real-world scenarios. 1.1. Our Contributions We present a novel system-theoretic framework for SLA management in reconfigurable multi-agent RAG systems, specifically tailored for question-answering applications. Our main contributions are as follows: • Dynamically Reconfigurable Horizontal Scaling Framework : We propose a method for horizontally scaling multi-agent RAG systems by replicating agents with diverse configurations. This approach systematically improves performance by enabling the system to dynamically adjust resource allocation based on the specific requirements of each query. • Implementation and Experimental Validation : We detail the architecture and implementation of our Abstractive Question Answering Intent Handler. Additionally, we present experimental results that demonstrate the effectiveness of our approach, including comparisons of strategies employed by different agent architectures. • Novel Metrics and Dataset Creation : We introduce novel metrics and a dataset creation process to address stylistic adherence requirements, which are critical for many industry users. By addressing the challenges of SLA management in multi-agent RAG systems, our work contributes to the development of QA systems that are adaptable, efficient, and capable of delivering high-quality answers under diverse operational conditions. 1.2. Related Work Scaling Retrieval-Augmented Generation (RAG) systems has been an active area of research, with most approaches focusing on vertical scaling—decomposing tasks into subtasks assigned to specialized components. Recent studies incorporate Large Language Models (LLMs) at the planning stages to mitigate issues like irrelevant retrieval and focus drift. For instance, PlanRAG uses an LLM to generate retrieval plans, enhancing context relevance ( Lee et al. 2024 ) . Similarly, Retrieve-Plan-Generate (RPG) separates planning and answering stages to improve long-form question answering ( Lyu et al. 2024 ) . AutoGPT+P introduces planning loops for step-by-step plan generation in dynamic environments ( Birr et al. 2024 ) . Horizontal scaling through multi-agent systems has also been explored, where multiple agents are synchronously assigned to the same question. LLM-Debate employs multiple LLM agents debating to reach better answers ( Du et al. 2023 ) . Chain-of-Thought Self-Consistency (CoT-SC) generates multiple reasoning paths and selects the most consistent answer ( Wang et al. 2023 ) . Li et al. demonstrate that increasing the number of agents with simple voting improves performance ( Li et al. 2024 ) . Temporal scaling techniques involve sequential interactions with LLMs to enhance question-answering performance on complex queries. Shao et al. use iterative querying to refine answers for complex information needs ( Shao et al. 2023 ) . Query refinement methods, such as RQ-RAG involve rewriting, decomposing, and disambiguating complex queries have also