Automated Testing of Task-based Chatbots: How Far Are We? DOI: XXXXXXX.XXXXXXX Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2018/06

Diego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani · 2026

Read Automated Testing of Task-based Chatbots: How Far Are We? DOI: XXXXXXX.XXXXXXX Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2018/06 by Diego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13072.txt.

Description

Task-based chatbots help users complete jobs through conversation, and testing them well means exercising a large conversational space. Traditional testing techniques fall short, and chatbot-specific methods from industry and research still show limits, such as simplistic test scenarios and weak oracles. Diego Clerissi and three co-authors run a confirmatory study to measure those limits directly.

They test leading chatbot testing techniques on a curated set of task-based chatbots from GitHub, built on popular commercial and open-source platforms. The study asks a plain question: how far are we, really? By testing the testers on real chatbots, the paper gives a grounded assessment of where chatbot testing stands and where its scenarios and oracles most need strengthening.

Catalog record

CREATORDiego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13072

Browse the full library · About the Tower of Babel