ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

Hassan Hamad, Yingru Xu, Liang Zhao, Wenbo Yan, Narendra Gyanchandani · 2025

Read ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems by Hassan Hamad, Yingru Xu, Liang Zhao, Wenbo Yan, Narendra Gyanchandani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17052.txt.

Introduction

Hassan Hamad leads four co-authors in catching tool-use mistakes by chatbots in a 2025 paper. Language models increasingly call outside tools during conversations. Wrong calls, wrong arguments, and misread results undermine reliability. Their framework, ToolCritic, diagnoses those errors systematically.

ToolCritic recognizes eight distinct tool-calling error types. Examples include calling a tool too early and misaligning its arguments. It gives targeted feedback to the main model afterward. The main model then revises its own response. A synthetic dataset trained the critic on these categories. Testing used a standard schema-guided dialogue collection. Tool-calling accuracy improved by up to thirteen percent over baselines. The gains beat both plain prompting and self-correction attempts.

Catalog record

CREATORHassan Hamad, Yingru Xu, Liang Zhao, Wenbo Yan, Narendra Gyanchandani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17052

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems Hassan Hamad, Yingru Xu, Liang Zhao, Wenbo Yan, Narendra Gyanchandani Abstract Tool-augmented large language models (LLMs) are increasingly employed in real-world applications, but tool usage errors still hinder their reliability. We introduce ToolCritic, a diagnostic framework that evaluates and improves LLM behavior in multi-turn, tool-augmented dialogues. ToolCritic detects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinterpretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic’s feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more robust LLM integration with external tools in real-world dialogue applications. Figure 1: ToolCritic evaluates every assistant response in a conversation, whether a tool was called or not (this is not shown in the diagram due to space limitations). If ToolCritic detects an assistant error, it will produce a reasoning “thought” which will be fed back to the assistant. The assistant then will revise its own response based on the received feedback and produce an updated response. If ToolCritic does not detect a mistake, the conversation continues without interference. Note that while we show the Assistant “Initial Response” for clarity, in a real-world scenario, this response will be hidden from the user and only the revised response will be used. 1 Introduction Large language models (LLMs) have made significant strides, becoming viable options for conversational agents. Beyond general-purpose conversations, there is a growing demand for these models to invoke external tools—such as booking APIs or search engines—to complete diverse task-oriented requests. This integration, often referred to as function-calling or tool-calling, significantly expands the scope of an LLM’s potential applications ( Huang et al. 2024b ; Qu et al. 2024 ) . Yet, reliable and accurate tool usage remains a major challenge: while modern LLMs can call external APIs, they frequently misidentify which tool to invoke, supply incorrect arguments, or even misinterpret the tool’s output ( Qu et al. 2025 ) . Function-calling has gained increasing attention in the development of foundational LLMs ( Dubey et al. 2024 ; AI 2024 ; OpenAI 2023 ; Anthropic 2024 ) . Still, as our results will show, even LLMs equipped with function-calling capabilities often struggle with effectively using tools in general. LLMs are known to be prone to hallucinations and reasoning mistakes ( Bang et al. 2023 ; Guerreiro et al. 2023 ; Ji et al. 2023 ) . Recent strategies to improve LLM reliability often center on intrinsic self-correction (where the LLM inspects and corrects its own reasoning) ( Madaan et al. 2023 ; Shinn et al. 2023 ) , or external feedback loops (where an external model or source identifies mistakes or provides feedback to the LLM) ( McAleese et al. 2024 ; Gou et al. 2024 ) . While self-correction shows promise, it remains inconsistent—LLMs often fail to identify their own errors reliably, leading to only incremental improvements ( Huang et al. 2024a ; Tyen et al. 2024 ) . While these previous studies have focused on general reasoning errors, our work focuses on the function-calling ability of LLMs in multi-turn task-oriented dialogue (TOD) applications. Utilizing LLMs in multi-turn TOD scenarios, such as MultiWOZ ( Budzianowski et al. 2018 ) , SGD ( Rastogi et al. 2020 ) , and ToolTalk ( Farn and Shin 2023 ) datasets, presents unique challenges. At each turn, the LLM must determine whether a tool call is necessary, which requires tracking user intent over multiple turns, identifying the correct moments for tool invocation, selecting the appropriate tool from a potentially large set, and accurately using the tool’s specifications. This setup is prone to a wide range of errors and failure modes, more so than in general conversational dialogue. In this work, we identify and categorize eight distinct error types related to tool usage, such as premature tool invocation—where an LLM calls a tool before gathering all necessary information from the user—and observation-reasoning errors, where the LLM correctly calls a tool and receives an output but misinterprets this output, leading to an incorrect response. Motivated by findings that LLMs struggle with self-correction, we propose ToolCritic : a model that detects errors in LLM tool usage during multi-turn TODs and provides descriptive feedback that the LLM can use to revise and improve its responses (see examples in Figure 1 ). Our approach works as follows: In a dialogue between a user (human) and an assistant (an LLM), at each turn, ToolCritic inspects the assistant’s proposed response. If an error is detected, ToolCritic provides targeted, textual feedback describing the mistake. The assistant LLM, assumed to have strong in-context learning capabilities, uses this feedback to revise its response and correct the error on-the-fly. Our results confirm that self-correction is not viable in this context and may even degrade performance. Unlike previous work that broadly examines LLM reasoning errors or focuses on the function-calling ability of LLMs in isolation, we specifically investigate LLM function-calling in conversational dialogue. Our contributions can be summarized as follows: • An error taxonomy of eight distinct failure modes for LLMs calling external tools in multi-turn dialogues. • A diagnostic model (ToolCritic) trained on a synthetic dataset to detect errors and provide descriptive feedback for the LLM-as-Tool architecture in conversational dialogue. • A feedback loop mechanism that demonstrates up to a 13\% improvement in dialogue success rates over self-correction and zero-shot approaches on the Schema Guided Dialogue (SGD) dataset ( Rastogi et al. 2020 ) . The remainder of this paper is structured as follows: Section 2 provides some background and reviews related work to highlight the differences from our approach. Section 3 describes the pipeline to generate the synthetic dataset needed for fine-tuning. Section 4 discusses the training of ToolCritic. Section 5 presents the experimental results. We conclude in Section 6 . Figure 2: Each error category defines a specific mistake, or failure mode, that is common for LLMs when interacting with tools in multi-turn dialogue. This granularity ensures the resulting diagnostic model, ToolCritic, can provide very specific feedback for the assistant LLM. 2 Background and Related Work The integration of tools with large language models (LLMs), also referred to as function-calling, has gained significant attention in recent years, driven by the enhanced reasoning capabilities of these models. Since LLMs are primarily pretrained on language modeling tasks, additional post-training is helpful to improve their function-calling abilities. Toolformer ( Schick et al. 2023 ) employs a self-supervised training approach to enhance a model’s proficiency in tool invocation. In ( Patil et al. 2023 ) , the authors developed a large-scale dataset of user prompts and API calls, followed by self-instruct fine-tuning of a LLaMA-7B-based model, which significantly improved API invocation accuracy. As discussed in the introduction, LLMs are prone to reasoning errors and hallucinations ( Bang et al. 2023 ; Guerreiro et al. 2023 ; Ji et al. 2023 ) , and these issues are often

Read the complete text