Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection
Yongqiang Chen, Gang Niu, James Cheng, Bo Han, Masashi Sugiyama · 2025
Read Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection by Yongqiang Chen, Gang Niu, James Cheng, Bo Han, Masashi Sugiyama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20963.txt.
Introduction
Yongqiang Chen leads four co-authors on when and why multi agent debate among language models fails, and whether it truly underperforms a single agent. Debate was promoted as a way to pool model wisdom and even supervise stronger systems, yet mounting results show it often loses to simpler setups.
Two popular forms are analyzed by their incentives, competitive debate where agents hold opposing sides, and consensus seeking debate where agents are pushed to agree. Both forms carry structural flaws in the analysis, such as pressure that rewards winning or agreeing over being right. The findings guide when debate helps, when it hurts, and how protocols might be repaired.
Catalog record
| CREATOR | Yongqiang Chen, Gang Niu, James Cheng, Bo Han, Masashi Sugiyama |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20963
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection Yongqiang Chen, Gang Niu, James Cheng, Bo Han, Masashi Sugiyama Abstract Accurate detection of errors in large language models (LLM) responses is central to the success of scalable oversight, or providing effective supervision to superhuman intelligence. Yet, self-diagnosis is often unreliable on complex tasks unless aided by reliable external feedback. Multi-agent debate ( MAD ) seems to be a natural alternative to external feedback: multiple LLMs provide complementary perspectives and cross-checks for error detection. However, prior MAD protocols frame debate as a zero-sum game , where the debaters compete to win the game instead of seeking the truth. Consequently, it leads to debate hacking : debaters tend to mislead the judge by misinterpreting the task or presenting overconfident claims, which introduce more mistakes and underperform single-agent methods. To mitigate the issue, we introduce a new collaborative MAD protocol, termed ColMAD , that reframes MAD as a non-zero sum game . Specifically, ColMAD encourages multiple agents to criticize each other in a supportive way, such that they can complement the missing points of each other. Therefore, the judge agent can make a more informative conclusion based on more comprehensive evidence. Empirically, we show that ColMAD significantly outperforms previous competitive MAD by 19% and brings non-trivial improvements over single-agent methods in error detection. 1 Introduction Large language models (LLMs) have gained huge success in solving tasks at various complexity levels ( Bubeck et al. 2023 ; Jaech et al. 2024 ; Guo et al. 2025 ) . As LLMs grow more powerful and capable of challenging tasks that require human expert-level knowledge, it becomes harder for humans to effectively understand and supervise LLMs ( Amodei et al. 2016 ) . Hence, it is essential to seek scalable oversight that provides effective supervision signals to powerful LLMs beyond human intelligence ( Amodei et al. 2016 ; Christiano et al. 2018 ; Irving et al. 2018a ; Bowman et al. 2022 ; Burns et al. 2023a ; Khan et al. 2024 ; Kenton et al. 2024 ) . Detecting the errors in LLM responses is critical to the success of scalable oversight ( Tyen et al. 2024 ; Huang et al. 2024 ; Kamoi et al. 2024a ) . However, LLMs are shown to be struggling to self-diagnose their own mistakes when without reliable external feedback ( Kamoi et al. 2024a ) . In particular, Kamoi et al. 2024b found that powerful LLMs like GPT-4 can not reliably detect errors in the responses of GPT-4 or Llama-2. Figure 1: Comparison between competitive multi-agent debate ( CopMAD ) and collaborative multi-agent debate ( ColMAD ). Given an LLM response to a task of generating a math problem according to some given requirements, it is required to examine whether the LLM response meets all the requirements properly. The original MAD scheme suffers from debate hacking . Due to the zero-sum nature, the dishonest debaters tend to mislead the judge with misinterpreted pieces of evidence (”including the cost of the scooter is unnecessary”) or overconfident claims (”fundamentally flawed”). Instead, collaborative debate aims to complement the missing information of each other in order to assist the judge in making a more informed decision. To complement the requirement for external feedback in error detection, it is natural to incorporate feedback from other LLMs to help with error detection. As LLMs differ in their knowledge and error tendencies ( Kim et al. 2025 ) , they are unlikely to commit the same mistakes simultaneously, thereby providing complementary signals for error detection (see in Fig. 2 ). In fact, Multi-Agent Debate ( MAD ) is a promising scheme to realize this insight that incorporates the knowledge of multiple LLM agents to resolve complex reasoning tasks and improve over single-agent methods ( Chen et al. 2025 ; Feng et al. 2025 ; Buhl et al. 2025 ) . In particular, Khan et al. 2024 and Kenton et al. 2024 showed that even a weak LLM can easily identify the flaws and select the correct answer from the debate of powerful LLMs on complex tasks. Nevertheless, it remains underexplored whether MAD also facilitates the LLM error detection, and thus it raises an interesting research question: Can we incorporate MAD to help with LLM error detection? In this work, we investigate the feasibility of using MAD to detect errors in LLM responses. Despite the high potential of the MAD , we find that previous MAD approaches will even introduce more mistakes and underperform the use of a single LLM in error detection (see Fig. 3 ). Interestingly, as previous approaches often frame MAD as a zero-sum game where the debaters compete with each other, we find that stronger LLMs will exhibit debate hacking behaviors in competitive MAD ( CopMAD ): as shown in Fig. 1 , debaters will misinterpret the task requirements and present claims in an overconfident tone to mislead the judge agent. In other words, CopMAD debaters will try every possibility to persuade the judge to win the game , instead of providing honest and evidential statements . Consequently, CopMAD can degenerate or even underperform single-agent methods (Proposition 2.2 ). To mitigate the issue, we propose a new MAD protocol called Collaborative Multi-Agenet Debate ( ColMAD ) that reframes MAD as a non-zero sum game. In contrast to CopMAD , instead of prompting the agents to win the game, ColMAD asks debaters to collaborate and complement each other’s missing points. Thus, the judge agent could make a more informative and objective decision based on the debating transcripts (Proposition 2.3 ). Empirically, across three benchmarks in ( Kamoi et al. 2024b ) , we demonstrate that ColMAD can lead to up to 4% improvements in identifying mistakes of LLMs compared to single-agent-based methods. In contrast, CopMAD will lead to up to 15% performance decrease compared to single-agent methods. In addition to the increased correctness of the error detection, we also find that the explanations given by ColMAD to why there are errors in LLM responses are more aligned to humans . The more human-aligned MAD protocol provides useful insights for scalable oversight. 2 Collaborative Multi-Agent Debate In this section, we initiate an investigation of MAD for detecting errors in LLM responses. 2.1 Potential of Multi-Agent Collaboration As Kamoi et al. 2024b showed that a single LLM can hardly tell the errors in LLM responses when without no reliable external feedback, a natural idea is to investigate whether incorporating multiple LLMs can mitigate the issue. Specifically, we consider the collaboration of two LLMs in error detection through MAD , and our discussion can also be generalized to two or more LLMs. Specifically, we consider two agents A (Alice) and B (Bob), and denote the predictions of the error detections as \hat{{\bm{y}}}_{A} and \hat{{\bm{y}}}_{B} , with rationales (e.g., CoT reasoning) as x_{A} and x_{B} , respectively. During the debate, they will emit messages m_{A} and m_{B} to convince a judge J . Assumption 2.1 (Optimal Judge Strategy) . Denoting the label as Y\in\{0,1\} with prior probability \pi\in(0,1) , without debating, the judge will derive the final answer based on the initial responses X_{0}=(x_{A},x_{B}) . Assuming the judge J uses the Bayes test on the total