Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing

Álvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso · 2025

Read Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing by Álvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01541.txt.

Description

Alvaro Zaera and co-authors present a two-step method to catch off-topic requests in chat systems, in this 2025 paper. Task chatbots must reject queries outside their scope to stay safe and useful. Their system runs live in a product that handles tens of thousands of chats a day.

First, an uncertainty score flags doubtful cases from the normal intent sorter. Only those cases go to a fine-tuned large language model for a final call. That split keeps cost low while lifting accuracy. Tests on standard sets and on real off-topic data from the live system gave top results. The authors stress the balance of speed and care in the design.

Catalog record

CREATORÁlvaro Zaera, Diana Nicoleta Popa, Ivan Sekulić, Paolo Rosso
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01541

Browse the full library · About the Tower of Babel