Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing
Álvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso · 2025
Read Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing by Álvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01541.txt.
Description
Alvaro Zaera and co-authors present a two-step method to catch off-topic requests in chat systems, in this 2025 paper. Task chatbots must reject queries outside their scope to stay safe and useful. Their system runs live in a product that handles tens of thousands of chats a day.
First, an uncertainty score flags doubtful cases from the normal intent sorter. Only those cases go to a fine-tuned large language model for a final call. That split keeps cost low while lifting accuracy. Tests on standard sets and on real off-topic data from the live system gave top results. The authors stress the balance of speed and care in the design.
Catalog record
| CREATOR | Álvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-01541