CRMArena-Pro : Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Kung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal · 2025
Read CRMArena-Pro : Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions by Kung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18878.txt.
Description
Kung-Hsiang Huang and co-authors published a benchmark paper in 2025 testing how well AI agents handle real business software work. Customer relationship platforms run sales and service for most companies, yet realistic public test data is rare, so earlier benchmarks felt artificial. The team expanded an existing CRM testbed into a broader, more faithful suite.
Their benchmark holds nineteen expert-checked tasks spanning sales, service, and price quoting, in both business-to-business and consumer settings. Conversations run over multiple turns with varied personas, and keeping secrets is graded too. Top agents passed only about fifty-eight percent of single-turn tasks and thirty-five percent of multi-turn ones. Workflow execution scored highest, while confidentiality awareness sat near zero without special prompting. Enterprise readiness, the authors conclude, is not here yet.
Catalog record
| CREATOR | Kung-Hsiang Huang Akshara Prabhakar Onkar Thorat Divyansh Agarwal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18878