CRMArena-Pro : Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions

Kung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal · 2025

Read CRMArena-Pro : Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions by Kung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18878.txt.

Description

Kung-Hsiang Huang and co-authors published a benchmark paper in 2025 testing how well AI agents handle real business software work. Customer relationship platforms run sales and service for most companies, yet realistic public test data is rare, so earlier benchmarks felt artificial. The team expanded an existing CRM testbed into a broader, more faithful suite.

Their benchmark holds nineteen expert-checked tasks spanning sales, service, and price quoting, in both business-to-business and consumer settings. Conversations run over multiple turns with varied personas, and keeping secrets is graded too. Top agents passed only about fifty-eight percent of single-turn tasks and thirty-five percent of multi-turn ones. Workflow execution scored highest, while confidentiality awareness sat near zero without special prompting. Enterprise readiness, the authors conclude, is not here yet.

Catalog record

CREATORKung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18878

Browse the full library · About the Tower of Babel