Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence

Sumanth Balaji, Piyush Mishra, Aashraya Sachdeva, Suraj Agrawal · 2026

Read Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence by Sumanth Balaji, Piyush Mishra, Aashraya Sachdeva, Suraj Agrawal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00596.txt.

Description

Sumanth Balaji, Piyush Mishra, Aashraya Sachdeva, and Suraj Agrawal argue that customer support chatbots need a harder test than most benchmarks give them. Old phone-menu systems followed rigid scripts, while language model agents promise flexibility. The open question is whether an agent follows business policy through multi-step tasks, handles dependencies between steps, and stays on track when users behave unpredictably.

Their benchmark, JourneyBench, generates realistic support scenarios from graph representations of customer journeys. A new score, the User Journey Coverage Score, measures how completely an agent adheres to policy across a whole journey rather than on isolated turns. The work targets a gap between agents that merely complete tasks and agents a company can trust with real customers.

Catalog record

CREATORSumanth Balaji, Piyush Mishra, Aashraya Sachdeva, Suraj Agrawal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-00596

Browse the full library · About the Tower of Babel