Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation
Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, Srikant Panda · 2025
Read Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation by Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, Srikant Panda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02097.txt.
Introduction
Pattnayak and co-authors present a hybrid chatbot design for company support desks, published in 2025, meant to stay fast and accurate over long conversations. Retrieval-based chatbots ground their answers in company documents, but real deployments strain them. Questions vary wildly, answers lag, and the bot sometimes invents facts. Fast-changing product knowledge makes upkeep worse. The design splits traffic by confidence.
Simple, well-known questions get instant canned answers. Complex or ambiguous ones are routed to the retrieval pipeline, which composes a grounded reply. A context manager tracks the conversation across turns, and feedback loops let the routing adapt. The authors built and tested the framework on enterprise-style workloads. The pitch is balance: the speed of scripts where scripts suffice, the depth of search where they do not. Hallucinations fall because risky answers are gated.
Catalog record
| CREATOR | Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, Srikant Panda |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02097
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, Srikant Panda Abstract Retrieval-Augmented Generation (RAG) systems and large language model (LLM)-powered chatbots have significantly advanced conversational AI by combining generative capabilities with external knowledge retrieval. Despite their success, enterprise-scale deployments face critical challenges, including diverse user queries, high latency, hallucinations, and difficulty integrating frequently updated domain-specific knowledge. This paper introduces a novel hybrid framework that integrates RAG with intent-based canned responses, leveraging predefined high-confidence responses for efficiency while dynamically routing complex or ambiguous queries to the RAG pipeline. Our framework employs a dialogue context manager to ensure coherence in multi-turn interactions and incorporates a feedback loop to refine intents, dynamically adjust confidence thresholds, and expand response coverage over time. Experimental results demonstrate that the proposed framework achieves a balance of high accuracy (95%) and low latency (180ms), outperforming RAG and intent-based systems across diverse query types, positioning it as a scalable and adaptive solution for enterprise conversational AI applications. 1 Introduction Recent progress in NLP has drastically changed the landscape of conversational AI, and among such new state-of-the-art solutions, a class of Retrieval-Augmented Generation (RAG) systems has emerged. By combining large language models (LLMs) with separate information retrieval pipelines, RAG systems can generate contextually rich and factually grounded responses, which are necessary for knowledge-intensive applications Lewis et al. 2020 . However, enterprise-scale conversational AI systems often face real-world challenges such as diverse user query patterns, varying levels of query complexity, and stringent latency requirements for seamless user experiences. High computational costs, susceptibility to hallucinations when retrieval is misaligned, and inefficiencies in managing frequently updated domain-specific knowledge further compound these challenges, particularly in dynamic environments like customer support Sanh et al. 2020b ; Rocktäschel et al. 2020 . In practice, ensuring that such systems can scale while maintaining accuracy and low latency remains an industry pain point. In contrast, classical intent-based chatbots are efficient in processing frequently asked questions (FAQ) and other predictable queries, thanks to using pre-defined responses. Their lightweight computational footprint and scalability also make them well-suited for high-confidence, domain-specific scenarios Serban et al. 2017 ; Shah et al. 2018 . However, these systems are inherently rigid and often struggle with query diversity, especially when faced with ambiguous or context-dependent user interactions. In high-demand enterprise settings, the inability of intent-based systems to adapt quickly to evolving user needs or handle complex multi-turn dialogues Shah et al. 2018 ; Zhao 2020 results in inconsistent user experiences and increased operational overhead for manual updates. The inability to balance adaptability with efficiency underscores the need for hybrid systems that synergize the strengths of RAG and intent-based approaches. In order to solve these challenges, we propose a novel hybrid framework that combines RAG systems with intent-based canned responses for dynamic, multi-turn customer service interactions. While prior works have explored combining RAG and intent-based systems independently, our contribution lies in a cohesive framework that not only integrates these elements but also introduces a dynamic confidence-based routing mechanism refined through user feedback. This mechanism ensures that query routing decisions are continuously optimized based on real-time user interactions, enabling a system that evolves and adapts without manual intervention. Additionally, our framework addresses scalability challenges by efficiently balancing computational resources, making it particularly suited for enterprise-scale applications where latency and accuracy are paramount. Our approach utilizes a dynamic query routing mechanism that evaluates the intent confidence level of user queries: • High-confidence queries are resolved using predefined canned responses to ensure low latency and computational efficiency. • Low-confidence or ambiguous queries are routed to the RAG pipeline, enabling contextually enriched responses generated from external knowledge. The framework is further enhanced with a dialogue context manager, keeping track and managing evolving intents across multiple turns, ensuring consistent and coherent interactions. Additionally, a feedback loop continuously refines the intent repository, adapting to emerging user needs and expanding response coverage over time. Our system is designed to meet enterprise latency standards, delivering responses within an acceptable threshold (sub-200ms latency and high turn efficiency), thereby ensuring user engagement and satisfaction in real-time applications Pattnayak et al. 2024 . Our Contributions This work makes the following key contributions: 1. Hybrid Conversational Framework: We propose a novel architecture which combines RAG systems with intent-based canned responses; the queries are routed dynamically for optimizing response latency and computational cost without compromising accuracy. 2. Multi-Turn Dialogue Management: We introduce a dialogue context manager which can track the evolving user intents and guarantee coherence in responses over multiple turns, thus addressing a key gap in the current systems. 3. Feedback-Driven Adaptability: Our framework incorporates a feedback loop to enable continuous refinement of intents, canned responses and confidence thresholds, thereby improving system adaptability and coverage for real-world applications. 4. Comprehensive Evaluation: Extensive experiments on synthetic and real-world datasets demonstrate significant improvements in accuracy, latency, and cost efficiency compared to state-of-the-art baselines. 5. Real-World Applicability: Our framework is designed for enterprise-scale deployment, handling diverse user queries efficiently, from repetitive FAQs to complex knowledge-based questions, while adhering to industry latency standards critical for user retention. By addressing key challenges faced by enterprise conversational AI systems, such as query diversity, dynamic knowledge updates, and real-time latency requirements, our proposed framework offers a scalable, adaptive, and efficient solution. This work advances task-oriented dialogue systems, particularly in domains where multi-turn interactions and dynamic knowledge management are essential for operational success. 2 Related Work 2.1 Retrieval-Augmented Generation (RAG) Recent advancements in RAG have enhanced contextual retrieval and generative capabilities, improving incident resolution in IT support Isaza et al. 2024 , question-answering systems, and domain-specific chatbots Veturi et al. 2024 . Research on noise handling Cuconasu et al. 2024 ; Meghwani et al. 2025 and reinforcement learning Kulkarni et al. 2024 further optimizes RAG for precision and adaptability in complex applications. By retrieving relevant documents during inference, RAG systems mitigate common LLM challenges such as hallucinations and outdated knowledge Lewis et al. 2020 ; Sanh et al. 2020b . These systems are particularly effective for knowledge-intensive tasks where accuracy and factual grounding are critical Pattnayak et al. . Despite their effectiveness, RAG systems face significant challenges, including high computational costs and latency due to the dual retrieval and generation processes. Enterprise settings also pose unique challenges, such as diverse user queries, latency constraints, and evolving domain knowledge needs Lewis et al. 2020 ; Pattnayak et al. 2025c . Moreover, most existing RAG systems are optimized for single-turn interactions and struggle with maintaining coherence in multi-turn dialogues, where evolving user intents require dynamic retrieval and contextual adaptation Rocktäschel et al. 2020 . Recent efforts to optimize RAG pipelines, such as multi-stage retrieval systems Lee et al. 2020 and model distillation Sanh et al. 2020b , have reduced latency but do not address the complexities of multi-turn interactions Sanh et al. 2020a . 2.2 Intent-Based Chatbots Intent-based chatbots work well for predictable Pattnayak 2017 , high-confidence queries by mapping user inputs to predefined intents. These systems are widely used in domains like customer support, where they efficiently handle FAQs and repetitive queries with minimal computational overhead Serban et al. 2017 ; Shah et al. 2018 . However, their reliance on predefined intents