MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
Chaithanya Bandi, Ben Hertzberg, Geobio Boo, Tejas Polakam, Jeff Da, Sami Hassaan, Manasi Sharma, Andrew Park, Ernesto Hernandez, Dan Rambado, Ivan Salazar, Rafael Cruz, Chetan Rane, Ben Levin, Brad Kenstler, Bing Liu · 2026
Read MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers by Chaithanya Bandi, Ben Hertzberg, Geobio Boo, Tejas Polakam, Jeff Da, Sami Hassaan, Manasi Sharma, Andrew Park, Ernesto Hernandez, Dan Rambado, Ivan Salazar, Rafael Cruz, Chetan Rane, Ben Levin, Brad Kenstler, Bing Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00933.txt.
Description
Chaithanya Bandi and colleagues introduce MCP-Atlas, a benchmark for tool-use competency against production MCP servers. Existing evaluations miss realistic multi-step workflows, breadth across real servers rather than mocks, and structured claim-level scoring free of verbosity or style. MCP-Atlas holds 1,000 tasks by human experts spanning 36 real servers and 220 tools, with prompts specifying no servers, tools, or parameters. Agents must find relevant tools among plausible distractors and compose multi-step, cross-server workflows, scored against atomic claims from tool outputs. Answer-centric scoring credits valid alternative trajectories.
Catalog record
| CREATOR | Chaithanya Bandi, Ben Hertzberg, Geobio Boo, Tejas Polakam, Jeff Da, Sami Hassaan, Manasi Sharma, Andrew Park, Ernesto Hernandez, Dan Rambado, Ivan Salazar, Rafael Cruz, Chetan Rane, Ben Levin, Brad Kenstler, Bing Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61798 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00933