REIC: RAG-Enhanced Intent Classification at Scale DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Classification and regression trees CCS
Ziji Zhang, Michael Yang, Zhiyu Chen, Yingying Zhuang, Shu-Ting Pi, Qun Liu, Rajashekar Maragoud, Vy Nguyen, Anurag Beniwal · 2025
Read REIC: RAG-Enhanced Intent Classification at Scale DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Classification and regression trees CCS by Ziji Zhang, Michael Yang, Zhiyu Chen, Yingying Zhuang, Shu-Ting Pi, Qun Liu, Rajashekar Maragoud, Vy Nguyen, Anurag Beniwal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00210.txt.
Introduction
Ziji Zhang and co-authors present a method to sort customer requests at very large scale in work from 2025. Service centers must route each request to the right agent fast. As product lines grow, the list of request types grows and shifts by sector.
Their system pairs sorting with live search over stored knowledge. Fresh facts flow in at run time, so the sorter stays current without frequent retraining. Tests on real service data beat standard fine-tuned models and plain prompt methods. Gains held both inside the home sector and in new sectors. The approach suits firms whose request types change faster than models can be rebuilt.
Catalog record
| CREATOR | Ziji Zhang, Michael Yang, Zhiyu Chen, Yingying Zhuang, Shu-Ting Pi, Qun Liu, Rajashekar Maragoud, Vy Nguyen, Anurag Beniwal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00210
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
REIC: RAG-Enhanced Intent Classification at Scale DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Classification and regression trees CCS: Computing methodologies Natural language processing Ziji Zhang, Michael Yang, Zhiyu Chen, Yingying Zhuang, Shu-Ting Pi, Qun Liu, Rajashekar Maragoud, Vy Nguyen, Anurag Beniwal Abstract . Accurate intent classification is critical for efficient routing in customer service, ensuring customers are connected with the most suitable agents while reducing handling times and operational costs. However, as companies expand their product lines, intent classification faces scalability challenges due to the increasing number of intents and variations in taxonomy across different verticals. In this paper, we introduce REIC , a R etrieval-augmented generation E nhanced I ntent C lassification approach, which addresses these challenges effectively. REIC leverages retrieval-augmented generation (RAG) to dynamically incorporate relevant knowledge, enabling precise classification without the need for frequent retraining. Through extensive experiments on real-world datasets, we demonstrate that REIC outperforms traditional fine-tuning, zero-shot, and few-shot methods in large-scale customer service settings. Our results highlight its effectiveness in both in-domain and out-of-domain scenarios, demonstrating its potential for real-world deployment in adaptive and large-scale intent classification systems. 1. Introduction Customer service ( Cui et al. 2017 ; Yu et al. 2021a ; Nguyen et al. 2020 ; Qi et al. 2021 ; Parikh et al. 2023 ; Zhou et al. 2023 ; Chen et al. 2024 ) is critical for modern e-commerce but also one of the most resource-intensive departments. Different agents, either human or model, are trained to handle specific types of customer issues, making precise intent classification, particularly at the issue level, crucial for efficient routing. High issue-oriented intent accuracy ensures that customers are connected with the most suitable agents, reducing unnecessary transfers and lowering handling times. This optimization not only enhances customer satisfaction but also cuts operational costs by streamlining interactions and improving overall service efficiency. For model-based automatic resolvers in chatbot agentic systems ( Gupta et al. 2024 ; Chen et al. 2023 ) , the ability to precisely identify user intent is essential for delivering contextually appropriate and solution-oriented responses. As companies expand their product lines, intent classification faces two key challenges. First, the number of customer intents grows over time, requiring models to adapt to new intents quickly. Second, intent taxonomies can vary across product lines, making it difficult to maintain a unified classification system. For example we organize products into different verticals in e-commerce: with third-party products, customer usually inquire about physical retail orders or consumer accounts, and intents are categorized into three levels from coarse to fine-grained. In contrast, first-party products require more customized customer services due to our proprietary device and digital product offerings. For instance, a customer seeking device troubleshooting may interact with an agent who can access real-time diagnostic information and perform specific troubleshooting steps on the user’s behalf. This necessitates a broader set of intent categories to accommodate diverse customer needs, as illustrated in Figure 1 . This heterogeneity complicates intent classification, demanding scalable and flexible approaches to ensure accurate routing and efficient customer service. In this work we demonstrate our method using two verticals but it can be easily adapted to more. In this paper, we propose a novel R etrieval-augmented generation E nhanced I ntent C lassification ( REIC ) approach that reduces computational complexity and improves scalability for intent classification. We demonstrate the effectiveness of this approach through extensive experiments on real-world datasets , showing that our method outperforms traditional fine tuning and zero-shot or few-shot methods in large-scale customer service settings. Our results on both in-domain and out-of-domain intents demonstrate its potential to improve classification accuracy and enable dynamic updates without retraining, making it ideal for industry-scale applications. Figure 1. We present the heterogeneous intent structure with representative examples, illustrating the intent label hierarchy in each vertical. 2. Related Work 2.1. Intent classification Early work on intent classification for dialogues often relied on bag-of-words or recurrent models. For example, Schuurmans and Frasincar 2019 evaluated various classifiers on a multi-domain intent dataset and found that a simple SVM with hierarchical label taxonomy outperformed deeper LSTM models. With the advance of transformer architectures, researchers began to leverage self-attention and multi-task learning for intent understanding. Ahmadvand et al. 2020 introduced a joint intent mapping model that simultaneously classifies high-level intent and maps queries to fine-grained product categories. Wang et al. 2021 employed a slowly updated text encoder and global/local memory networks to mitigate catastrophic forgetting and parameter explosion for large-scale intent detection task. Recent work has pushed toward using large pre-trained models and retrieval-based prompting to enable cross-domain and zero/few-shot intent classification. Liu et al. 2024 proposed a framework which integrates a fine-tuned XLM-based intent classifier with an LLM to essentially treat multi-turn intent understanding as a zero-shot task. Yu et al. 2021b also explored retrieval-based methods for intent classification and slot filling tasks in few-shot settings. Our work adopts similar in-context learning (ICL) setup while focusing on handling large-scale multi-domain intent classification task from industry level applications. 2.2. In-context Learning The performance of LLM has been significantly enhanced in few-shot and zero-shot NLP tasks through ICL. Recent ICL research focus on how to effectively identify and interpret retrieved context. Guu et al. 2020 first showed how to pre-train masked language models with a knowledge retriever in an unsupervised manner. Karpukhin et al. 2020 proposed a training pipeline in which retrieval is implemented using dense representations alone and embeddings are learned from a small number of questions and passages with a dual-encoder. Ram et al. 2023 considered simple alternatives to only prepend retrieved grounding documents to the input, instead of modifying the LLM architecture to incorporate external information. Similar approaches have proven particularly effective in the application of RAG on dialogue systems ( Shuster et al. 2022b ; Shuster et al. 2022a ) , specifically goal-oriented and domain-specific dialogs from customer service scenarios ( Zhuang et al. 2021 ; Zhuang et al. 2023 ; Senel et al. 2024 ) . In our work, we utilize ICL in both LLM fine-tuning stage for data generation and at inference-time with an intent candidate retriever. Figure 2. The proposed REIC method from customer query to routing intent leveraging on vector retrieval and probability calculation. 3. Preliminary Intent classification for queries is typically framed as a multiclass text classification problem. Specifically, given a customer query q\in Q , the goal is to map it to one of the k pre-defined intents t\in T=\{t_{1},...,t_{k}\} using a model \mathcal{M} so that the predicted intent \hat{t}=\mathcal{M}(q) maximizes the probability of correctly classifying q . Formally, this can be expressed as: (1) \hat{t}=\arg\max_{t\in T}\,P(t\mid q;\theta) where P(t\mid q;\theta) denotes the probability of intent t given query q , parameterized by \theta of the model \mathcal{M} . Flat classification assumes independence among intent labels, which is often unrealistic in practice. Two key challenges arise at industry scale: 1) Scalability —a large number of labels makes flat classification inefficient and hard to maintain; 2) Label correlation —related intents (e.g., “Order Issue” vs. “Track Order”) are treated independently, ignoring useful hierarchical structure. To address this, we adopt hierarchical intent classification, where queries