Automated Testing of Task-based Chatbots: How Far Are We? DOI: XXXXXXX.XXXXXXX Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2018/06
Diego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani · 2026
Read Automated Testing of Task-based Chatbots: How Far Are We? DOI: XXXXXXX.XXXXXXX Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2018/06 by Diego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13072.txt.
Description
Task-based chatbots help users complete jobs through conversation, and testing them well means exercising a large conversational space. Traditional testing techniques fall short, and chatbot-specific methods from industry and research still show limits, such as simplistic test scenarios and weak oracles. Diego Clerissi and three co-authors run a confirmatory study to measure those limits directly.
They test leading chatbot testing techniques on a curated set of task-based chatbots from GitHub, built on popular commercial and open-source platforms. The study asks a plain question: how far are we, really? By testing the testers on real chatbots, the paper gives a grounded assessment of where chatbot testing stands and where its scenarios and oracles most need strengthening.
Catalog record
| CREATOR | Diego Clerissi, Elena Masserini, Daniela Micucci, Leonardo Mariani |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13072