AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?

Ranit Karmakar, Jayita Chatterjee · 2026

Read AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go? by Ranit Karmakar, Jayita Chatterjee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00334.txt.

Description

Ranit Karmakar and Jayita Chatterjee ask how much of an AI agent's work truly needs a frontier model. Working agents make many calls per request, and most are short, structured, and routine. Their benchmark, AgentFloor, arranges thirty tasks as a six-tier ladder, from following instructions through tool use to long-horizon planning under lasting constraints.

They ran sixteen open-weight models, from very small to mid-sized, alongside a leading closed model, over more than sixteen thousand scored runs. Small and mid-sized open models already handle much of the routine base, and the strongest open model matched the closed one overall while running cheaper and faster. The clear gap sits at long-horizon planning, where neither side was truly reliable. The design lesson is to reserve frontier models for the narrow tier that demands them.

Catalog record

CREATORRanit Karmakar, Jayita Chatterjee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00334

Browse the full library · About the Tower of Babel