AutomationBench
Daniel Shepard, Robin Salimans · 2026
Read AutomationBench by Daniel Shepard, Robin Salimans (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18934.txt.
Description
Shepard and Salimans introduce AutomationBench, a benchmark for AI agents on cross application workflows. Real workflows demand coordination, API discovery, and policy adherence across a CRM, inbox, calendar, and messaging platform. Tasks span Sales, Marketing, Operations, Support, Finance, and HR. Agents must find endpoints alone, follow business rules, and navigate misleading records. Grading is automated and end state only, and even the best models score below 10 percent.
Catalog record
| CREATOR | Daniel Shepard, Robin Salimans |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18698 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18934