ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox

Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen · 2026

Read ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox by Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10787.txt.

Description

Yuanyang Li and colleagues bring in ComplexMCP, a test bed rating language-model agents in live, linked, large-scale sandboxes. Today's agents cope with lone API calls but trip on the last mile of shop software automation, where tools are atomic, linked, and prone to noise. Built on the Model Context Protocol, ComplexMCP gives over 300 vetted tools from seven stateful sandboxes spanning office suites to cash systems. It uses a seed-led build to fake live site states and flaky API fails for firm, varied tests.

Tests across full-context and retrieval styles show a gap: even top-tier models fail to top 60 percent wins, far short of human scores above 90 percent. Fine-grained path study finds three snags: tool find saturation as action spaces grow, and over-trust where agents skip key site checks. Planned defeatism, a bent to talk away failure, not chase a fix, is the third.

Catalog record

CREATORYuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103947
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10787

Browse the full library · About the Tower of Babel