ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen · 2026
Read ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox by Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10787.txt.
Description
Yuanyang Li and colleagues bring in ComplexMCP, a test bed rating language-model agents in live, linked, large-scale sandboxes. Today's agents cope with lone API calls but trip on the last mile of shop software automation, where tools are atomic, linked, and prone to noise. Built on the Model Context Protocol, ComplexMCP gives over 300 vetted tools from seven stateful sandboxes spanning office suites to cash systems. It uses a seed-led build to fake live site states and flaky API fails for firm, varied tests.
Tests across full-context and retrieval styles show a gap: even top-tier models fail to top 60 percent wins, far short of human scores above 90 percent. Fine-grained path study finds three snags: tool find saturation as action spaces grow, and over-trust where agents skip key site checks. Planned defeatism, a bent to talk away failure, not chase a fix, is the third.
Catalog record
| CREATOR | Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103947 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10787