Stay Focused: Problem Drift in Multi-Agent Debate

Jonas Becker, Lars Benedikt Kaesberg, Andreas Stephan, Jan Philip Wahle, Terry Ruas, Bela Gipp · 2025

Read Stay Focused: Problem Drift in Multi-Agent Debate by Jonas Becker, Lars Benedikt Kaesberg, Andreas Stephan, Jan Philip Wahle, Terry Ruas, Bela Gipp (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19559.txt.

Description

Jonas Becker and co-authors study how groups of language models, debating a problem in turns, slowly wander away from the question they were asked, in a 2025 paper. Such debates often solve hard problems well, but their edge fades on problems that need long chains of thought. The authors name the wandering 'problem drift' and measure it across ten tasks of several kinds.

Open-ended writing tasks drifted most, in up to nine cases out of ten, while hard knowledge and reasoning tasks drifted far less. Eight human experts then read 170 drifting debates. The top causes were stalled progress, poor-quality replies, and unclear statements, in roughly a third, a quarter, and a quarter of cases. The team built a judge tool that spots drift and a policy that repairs about a third of the cases. The work explains one reason longer debates can hurt rather than help.

Catalog record

CREATORJonas Becker, Lars Benedikt Kaesberg, Andreas Stephan, Jan Philip Wahle, Terry Ruas, Bela Gipp
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE127 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19559

Browse the full library · About the Tower of Babel