AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?
Ranit Karmakar, Jayita Chatterjee · 2026
Read AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go? by Ranit Karmakar, Jayita Chatterjee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00334.txt.
Description
Ranit Karmakar and Jayita Chatterjee ask how much of an AI agent's work truly needs a frontier model. Working agents make many calls per request, and most are short, structured, and routine. Their benchmark, AgentFloor, arranges thirty tasks as a six-tier ladder, from following instructions through tool use to long-horizon planning under lasting constraints.
They ran sixteen open-weight models, from very small to mid-sized, alongside a leading closed model, over more than sixteen thousand scored runs. Small and mid-sized open models already handle much of the routine base, and the strongest open model matched the closed one overall while running cheaper and faster. The clear gap sits at long-horizon planning, where neither side was truly reliable. The design lesson is to reserve frontier models for the narrow tier that demands them.
Catalog record
| CREATOR | Ranit Karmakar, Jayita Chatterjee |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.00334