Enhanced Urdu Intent Detection with Large Language Models and Prototype-Informed Predictive Pipelines

Faiza Hassan, Summra Saleem, Kashif Javed, Muhammad Nabeel Asim, Abdur Rehman, Andreas Dengel · 2025

Read Enhanced Urdu Intent Detection with Large Language Models and Prototype-Informed Predictive Pipelines by Faiza Hassan, Summra Saleem, Kashif Javed, Muhammad Nabeel Asim, Abdur Rehman, Andreas Dengel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07857.txt.

Introduction

Faiza Hassan and five colleagues build a better intent detector for Urdu, the world's tenth most spoken language. Intent detection tells what a user wants from a sentence, a core skill for chatbots and search. Urdu has lacked tools that learn from just a few examples.

Their pipeline, called LLMPIA, first re-trains language models on unlabeled Urdu text with contrastive learning. A prototype-informed attention step then helps classify new intents from few examples. The team tested six language models and thirteen ways to compute similarity. On two public data sets, the system reached strong F1 scores in one-shot and five-shot settings. A case study rounds out the results.

Catalog record

CREATORFaiza Hassan, Summra Saleem, Kashif Javed, Muhammad Nabeel Asim, Abdur Rehman, Andreas Dengel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07857

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhanced Urdu Intent Detection with Large Language Models and Prototype-Informed Predictive Pipelines Faiza Hassan, Summra Saleem, Kashif Javed, Muhammad Nabeel Asim, Abdur Rehman, Andreas Dengel Abstract Multifarious intent detection predictors are developed for different languages, including English, Chinese and French, however, the field remains underdeveloped for Urdu, the 10 th 10^{\text{th}} most spoken language. In the realm of well-known languages, intent detection predictors utilize the strategy of few-shot learning and prediction of unseen classes based on the model training on seen classes. However, Urdu language lacks few-shot strategy based intent detection predictors and traditional predictors are focused on prediction of the same classes which models have seen in the train set. To empower Urdu language specific intent detection, this introduces a unique contrastive learning approach that leverages unlabeled Urdu data to re-train pre-trained language models. This re-training empowers LLMs representation learning for the downstream intent detection task. Finally, it reaps the combined potential of pre-trained LLMs and the prototype-informed attention mechanism to create a comprehensive end-to-end LLMPIA intent detection pipeline. Under the paradigm of proposed predictive pipeline, it explores the potential of 6 distinct language models and 13 distinct similarity computation methods. The proposed framework is evaluated on 2 public benchmark datasets, namely ATIS encompassing 5836 samples and Web Queries having 8519 samples. Across ATIS dataset under 4-way 1 shot and 4-way 5 shot experimental settings LLMPIA achieved 83.28% and 98.25% F1-Score and on Web Queries dataset produced 76.23% and 84.42% F1-Score, respectively. In an additional case study on the Web Queries dataset under same classes train and test set settings, LLMPIA outperformed state-of-the-art predictor by 53.55% F1-Score. Journal: Nuclear Physics B † † highlights: Introduces a contrastive learning approach to re-train pre-trained language models (PLMs) using unlabeled Urdu data, thus improve their ability to capture semantic nuances. Proposes an end-to-end few-shot learning framework, LLMPIA that combines pre-trained LLMs with a prototype-informed attention mechanism to predict unseen intent classes. Evaluates six pre-trained language models alongside their re-trained versions to measure the impact of contrastive learning within the LLMPIA pipeline. Benchmarks one Urdu-specific and five multilingual PLMs to determine the optimal model choice for Urdu intent detection applications. Investigates thirteen distinct similarity measures to identify the most effective approach for prototype-informed intent detection. 1 Introduction In the landscape of natural language processing (NLP), intent detection is an essential task to recognize user’s needs or purposes from their communication [ 13 ] . Primarily, intent detection is an integral part of numerous NLP applications such as virtual assistants [ 9 , 69 ] , chatbots [ 14 , 50 , 36 , 35 ] , and automated customer support systems [ 71 ] , etc. However, unlike many traditional NLP tasks such as text classification [ 4 , 7 , 46 , 56 , 26 , 57 , 37 , 1 , 43 , 54 ] , sentiment analysis [ 45 , 22 , 25 , 10 , 48 ] , hate speech detection [ 44 ] , paraphrasing [ 68 ] and name entity recognition [ 38 , 39 , 73 ] , intent detection poses significant challenges because of the dynamic nature of human intentions. While significant advancements have been made in development of intent detection applications for high-resource languages like English [ 29 , 41 , 74 , 77 , 23 , 11 ] , there is a pressing need to extend this capability to Urdu [ 42 ] , one of the most widely used languages in South Asia. Around the world, Urdu has over 170 million speakers and is the national language of Pakistan 1 1 1 https://mdcresearch.net/general-multicultural-resources-for-providers/information-for-providers-who-work-with-families-parents-who-read-chinese/ . Similar to other languages, Urdu language-specific intent detection applications are accompanied by significant challenges due to numerous factors such as individuals expressing their intentions in a variety of ways, influenced by their personality, knowledge, experiences and contextual factors [ 58 ] . A potential challenge is compiling a comprehensive intent detection dataset that covers all possible categories is nearly impossible, leading to the challenge of handling unseen intent categories. In advanced languages, the data annotation problem is addressed by designing predictors capable of identifying samples belonging to unseen categories based on training with samples from known categories [ 51 , 58 , 59 ] . In contrast, current Urdu language intent detection predictors only assign intent categories to samples that fall within the categories present in their training dataset. In Urdu language, another major challenge is to generate a comprehensive and semantically rich statistical representation of the text [ 78 ] . While language-specific and multilingual pre-trained language models [ 8 , 15 , 19 , 30 , 51 , 64 ] offer a foundation for Urdu NLP applications, their general training necessitates specialized re-training to enhance performance for specific Urdu language tasks. This study aims to design a unique, end-to-end Urdu intent detection predictive pipeline, that addresses the aforementioned challenges through the following key contributions: 1. With an aim to generate a comprehensive and semantically rich statistical representation of Urdu text, it presents a unique contrastive learning [ 16 ] approach to re-train pre-trained language models (PLMs) [ 47 ] using unlabeled Urdu data. The contrastive learning approach enables the model to learn words representation by distinguishing subtle similarities between text pairs based on their semantic meaning. 2. It presents a unique framework that encompasses end-to-end few-shot learning strategy based Urdu intent detection predictive pipelines (LLMPIA) that synergistically combines the strengths of pre-trained LLMs with a prototype-informed attention mechanism. LLMPIA is capable of predicting unseen intent classes based on its learning from seen intent classes. 3. To assess the effectiveness of the proposed contrastive learning re-training strategy within the LLMPIA pipeline, this study systematically compares the performance of six pre-trained language models against their re-trained versions. 4. To address the critical question of whether multilingual or Urdu language-specific pre-trained language models are more effective for Urdu intent detection, this research benchmarks the performance of one Urdu-specific and five multilingual models. This comparative analysis provides valuable insights into the optimal model selection for Urdu intent detection application development. 5. Within the realm of few-shot intent detection, it performs an extensive analysis of thirteen different similarity computation methods, aiming to identify the most optimal similarity measure for developing highly accurate prototype-informed intent detection predictors, providing a solid foundation for future research in few-shot learning for Urdu. 2 Related Work AI-driven intent detection applications landscape has witnessed development of both multilingual and language-specific predictors. Multilingual predictors offer the advantage of broad applicability across various languages but they struggle in capturing the fine-grained syntactic and semantic details that are necessary to identify users intent. In contrast, language-specific predictors are exclusively trained on a particular language and manage to produce better performance by acquiring a deep understanding of that language’s specific linguistic characteristics. Existing multilingual intent detection predictors exhibit significant variations in their linguistic scope. For instance, the predictor developed by Atuhurra et al. [ 8 ] targets 4 specific languages (Urdu, English, Japanese, and Swahili) while Cheng et al. [ 15 ] and Qu et al. [ 51 ] predictors are developed to handle 9 (Hindi, Turkish, Spanish, German, Japanese, Chinese, Portuguese, French, Thai) and 2 (English, Chinese) languages respectively. Conversely,

Read the complete text