UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen · 2026

Read UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents by Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11557.txt.

Introduction

Liang and seven colleagues present UniToolCall, a unified framework for tool learning in LLM agents. It sets one standard for toolsets, datasets, and tests, curating over 22,000 tools and a hybrid set of over 390,000 cases from 10 public datasets plus synthetic runs. An Anchor Linkage mechanism ties turns together, and seven public benchmarks are recast in a unified Query-Action-Observation-Answer form. Fine-tuned Qwen3-8B reaches 93.0 percent single-turn Strict Precision under Hybrid-20 conditions, beating rival models.

Catalog record

CREATORYijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54178
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11557

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen Abstract Tool-use capability is a fundamental component of LLM agents, enabling them to interact with external systems through structured function calls. However, existing research exhibits inconsistent interaction representations, largely overlooks the structural distribution of tool-use trajectories, and relies on incompatible evaluation benchmarks. We present UniToolCall, a unified framework for tool learning that standardizes the entire pipeline from toolset construction and dataset generation to evaluation. The framework curates a large tool pool of 22k+ tools and constructs a hybrid training corpus of 390k+ instances by combining 10 standardized public datasets with structurally controlled synthetic trajectories. It explicitly models diverse interaction patterns, including single-hop vs. multi-hop and single-turn vs. multi-turn, while capturing both serial and parallel execution structures. To support coherent multi-turn reasoning, we further introduce an Anchor Linkage mechanism that enforces cross-turn dependencies. Furthermore, we convert 7 public benchmarks into a unified Query–Action–Observation–Answer (QAOA) representation with fine-grained evaluation at the function-call, turn, and conversation levels. Experiments show that fine-tuning Qwen3-8B on our dataset substantially improves tool-use performance. Under the distractor-heavy Hybrid-20 setting, UniToolCall achieves 93.0% single-turn Strict Precision, outperforming commercial models including GPT, Gemini, and Claude. 1 1 1 https://github.com/EIT-NLP/UniToolCall . 1 Introduction Figure 1: Existing datasets are severely limited by the fragmentation problem. To address these challenges, UniToolCall introduces a standardized framework that provides robust structural constraints, yielding SOTA over strong baselines. The emergence of LLM agents marks a shift from passive text generation to goal-directed interaction with external environments ( 6 ; 16 ; 27 ) . A key capability underlying this shift is tool use, which enables agents to take actions by translating natural language instructions into executable function calls. Through tool use, LLM agents can access external knowledge, invoke APIs, and perform multi-step operations, extending their capabilities beyond parametric knowledge ( 28 ; 39 ) . Consequently, an agent’s effectiveness largely depends on its ability to select, compose, and execute tools reliably, making tool learning a central problem in agent research ( 23 ; 15 ) . In the current data-driven paradigm, progress in tool learning is largely determined by the availability and quality of training data, particularly tool-use trajectories that capture how agents interact with external environments. Early efforts such as ToolLLM ( 26 ) , ToolBench ( 25 ) , and API-Bank ( 13 ) construct such data by executing real-world APIs. While providing realistic supervision signals, they suffer from limited scalability and instability due to their reliance on external systems. To address these limitations, more recent works have shifted toward synthetic data generation, building simulated tool environments and automatically generating interaction trajectories (e.g., ToolForge ( 4 ) , LoopTool ( 40 ) , ASTRA ( 31 ) ). In parallel, a number of benchmarks have been proposed to evaluate tool-use capability, including ComplexFuncBench ( 41 ) , HammerBench ( 33 ) , and ACEBench ( 3 ) . Despite this progress, existing efforts are largely developed in isolation, leading to a fundamental fragmentation problem in tool learning. This fragmentation manifests along three key dimensions. First, representation inconsistency : different datasets adopt incompatible schemas to encode tool calls, arguments, and observations, making joint training across sources difficult. Second, structural under-modeling : current pipelines largely overlook the diversity of execution structures, particularly the distinction between serial and parallel tool invocation patterns. Third, evaluation mismatch : existing benchmarks rely on disparate protocols, tool definitions, and evaluation scripts, preventing fair and reproducible cross-dataset comparisons. Together, these issues hinder both scalable training and systematic evaluation of tool-use capabilities. To address these limitations, we propose UniToolCall , a unified framework for tool learning that standardizes the entire pipeline from toolset construction and data generation to evaluation under a shared representation. We first curate a large-scale tool pool by aggregating tools from multiple sources, resulting in a filtered set of over 22K tools. Building on this, we construct a hybrid training corpus that combines standardized public datasets with structurally controlled synthetic trajectories, yielding 390K instances spanning single-hop, multi-hop, single-turn, and multi-turn interactions. Crucially, our synthetic pipeline explicitly models both serial and parallel execution structures, enabling fine-grained analysis of execution patterns. Finally, we unify all data into a Query–Action–Observation–Answer (QAOA) representation 2 2 2 A single-hop sample is shown in Appendix C.2 . and introduce a standardized evaluation protocol with comprehensive metrics, enabling consistent and fair comparison across diverse settings. Our contributions are as follows: • Structurally-Aware data generation We propose a synthetic data generation pipeline that provides controlled supervision for single/multi-hop and single/multi-turn interactions. The pipeline explicitly models both serial and parallel execution patterns and introduces an Anchor Linkage mechanism to enforce cross-turn dependencies. • A standardized unified benchmark We convert heterogeneous public benchmarks into a unified QAOA format with shared matching rules and metrics, enabling fine-grained evaluation across function-call, turn, and conversation levels and fair comparison across diverse task structures. • Strong empirical Performance Fine-tuning a lightweight Qwen3-8B model on our framework achieves state-of-the-art results. Under the distractor-heavy Hybrid-20 setting, UniToolCall attains 93.0% single-turn Strict Precision, outperforming leading commercial models including GPT, Gemini, and Claude. 2 Related work Synthetic data generation Early work ( 30 ) generates instruction-style tool-use examples to teach basic API usage. Subsequent pipelines further automate dataset construction ( 14 ; 4 ; 40 ) , synthesizing tool-use trajectories at larger scale. Despite this, most generated trajectories tend to follow relatively simple interaction patterns. Moreover, the balance between serial and parallel tool execution is rarely considered. In contrast, our synthetic pipeline explicitly models structural diversity by considering both serial and parallel execution patterns across four interaction structures. Tool-use benchmarks Some studies rely on real environments ( 26 ; 34 ; 8 ) , where models interact with external tools through actual execution. To improve reproducibility, several benchmarks evaluate tool usage through simulated invocation while retaining real tool definitions ( 3 ; 21 ) . However, these benchmarks adopt heterogeneous schemas, evaluation rules, and task structures, which hinder fair comparison. To address these limitations, we construct a unified benchmark that evaluates heterogeneous datasets under a shared QAOA representation, enabling multi-granularity evaluation and providing a more comprehensive assessment of tool learning. Figure 2: The overall architecture of UniToolCall, comprising several interconnected modules: (1) Toolset construction; (2) Unified data synthesis engine; (3) Structural integration; (4) Evaluation protocol. 3 UniToolCall In this section, we present UniToolCall, a unified framework for tool learning. At the core of our framework is a standardized QAOA representation, which provides a consistent format for modeling tool interactions across datasets. As illustrated in Figure 2 , the framework consists of three components: a curated toolset , a data generation pipeline , and a structural assembly stage. In addition, we introduce a unified benchmark to enable consistent evaluation across tool-use scenarios. 3.1 Toolset construction To serve as the candidate pool for dataset construction, we construct a comprehensive toolset after applying the multi-stage filtering mechanism (Appendix A.3 ), denoted as \mathcal{T} . As illustrated in Figure 7 , the toolset is formed from three primary sources: (1) Academic benchmarks, (2) MCP servers, and (3) Constructed datasets. All tools are standardized into a unified

Read the complete text