Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate

Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu, Shuyuan Zheng, Jianbin Qin Beijing Institute of Technology, Zhuhai, University of Osaka, Shenzhen University [email protected], [email protected] · 2026

Read Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate by Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu, Shuyuan Zheng, Jianbin Qin Beijing Institute of Technology, Zhuhai, University of Osaka, Shenzhen University [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00820.txt.

Introduction

Hao and co-authors pull apart why AI agents change their minds in debate in this 2026 paper. Multi-agent debate is often judged by how much answers shift or agree. But a shift alone cannot show that peers helped. The team runs five GPT-4o agents on hard MMLU-Pro questions where they first disagree.

Three setups are matched: self-reflection alone, peer stances only, and full peer reasoning. Even with no peers, 37 percent of agent-question pairs flip. Among pairs stable under that control, 29.3 percent adopt a peer answer once stances are shown. Those adoptions mix fixes, new errors, and swaps between wrong answers. Full reasoning lifts each agent's own score but adds no extra group gain over self-reflection. It also burns 57 percent more tokens. A neutral nudge cuts both harmful and helpful adoption, with no clear accuracy gain. The claim: convergence is a murky signal of value.

Catalog record

CREATORXiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu, Shuyuan Zheng, Jianbin Qin Beijing Institute of Technology, Zhuhai, University of Osaka, Shenzhen University [email protected], [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00820

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu, Shuyuan Zheng, Jianbin Qin Beijing Institute of Technology, Zhuhai, University of Osaka, Shenzhen University [email protected], [email protected] Abstract Multi-agent debate (MAD) is a promising strategy for improving LLM reasoning, but when agents converge on a shared answer, it is unclear whether that convergence reflects genuine deliberation or social compliance. We show that the conventional answer flip rate conflates three distinct mechanisms: spontaneous instability, stance-induced conformity, and reasoning-induced persuasion. Our three-source decomposition framework isolates each through controlled counterfactual conditions. In the primary MMLU-Pro setting, 37% of agent-question observations change under self-reflection alone, while robustness tests show substantial model-dependent instability across GPQA-Diamond and three model families; strict conformity is 29% in the primary setting and remains predominantly harmful across model replications (57–77% correct-to-wrong). A controlled information-gradient experiment reveals that even vacuous reasoning is associated with 20–39% error adoption among resistant agents, with reasoning-like presentation carrying substantial persuasive weight. Harmful conformity can be predicted from Round 0 features (AUC = 0.79), and risk-targeted intervention reduces it by 13.6 percentage points ( p < 0.001 p<0.001 ). However, without correctness labels or self-reflection controls, reducing peer adoption does not improve accuracy, because harmful and beneficial influence cannot be distinguished. 2 2 footnotetext: Corresponding authors. † † footnotetext: Our source code is available at https://github.com/47hao030808/Not-All-Flips-Are-Conformity . 1 Introduction Consider the following scenario. Five language model agents independently answer a question; three happen to be wrong, two are correct. In a subsequent round of debate, one of the correct agents abandons its answer and aligns with the incorrect majority. The standard interpretation is clear: the agent was persuaded by its peers. But was it? The answer is less obvious than it appears. Had the same agent simply been asked to reconsider its answer in isolation, with no knowledge of its peers, it might have changed its mind anyway. Models are not perfectly stable reasoners; they exhibit what we term spontaneous instability , a tendency to revise answers upon re-examination even without new information. We call this change spontaneous because no external peer information is introduced, though the reconsideration itself is triggered by the experimental prompt. If we cannot separate this baseline drift from genuine social influence, then every measurement of conformity in multi-agent debate carries an unknown margin of error. This paper addresses a deceptively simple question: what actually drives stance convergence in MAD? We decompose observed answer changes into three sources: (1) spontaneous change that would occur regardless of peer input, (2) stance-induced conformity triggered by peer positions alone, and (3) additional change induced by exposure to peer reasoning. The decomposition is operationalized through a controlled experimental design in which each agent-question pair is observed under three counterfactual conditions (self-reflection, stance-only exposure, and full reasoning exposure), allowing us to attribute each observed change to its proximate cause. Using this decomposition, we find that spontaneous instability is a large baseline source of answer change (37% of agent-question pairs change under self-reflection alone), strict conformity is substantially lower than raw flip-rate estimates and is predominantly harmful (63.6% correct-to-wrong in the primary setting), and peer reasoning often increases error adoption rather than correcting it 10 ; 16 ; 24 . We further show that harmful conformity is partially predictable from Round 0 features and can be reduced by targeted intervention in a diagnostic setting, while a deployment-style intervention reduces peer adoption without improving accuracy. The paper is organized around three questions: measure ( RQ1 : decomposing stance convergence), explain ( RQ2 : reasoning form vs. content), and mitigate ( RQ3 : prediction and intervention). Our contributions are: • A three-source decomposition framework tested primarily on GPT-4o, with robustness replications across two additional model families and four benchmarks (MMLU-Pro, GPQA-Diamond, BIG-Bench Extra Hard (BBEH) 15 ; 12 ; 14 , AGIEval 36 ). • Empirical evidence that spontaneous degradation and harmful conformity are both predominantly accuracy-reducing, with spontaneous degradation the larger source in the primary GPT-4o setting. • A controlled information-gradient experiment showing that reasoning-like presentation leads to 20–39% error adoption among resistant agents, even when the reasoning contains no valid logical content. • A prediction model (ROC-AUC = 0.79) enabling risk-targeted intervention that reduces harmful conformity by 13.6 pp ( p<0.001 ), alongside a deployment variant demonstrating that peer-adoption reduction without correctness feedback does not improve accuracy. The remainder of the paper is organized as follows. Section 2 reviews related work. Section 3 describes the three-source decomposition and experimental design. Sections 4 – 6 present results for measurement, explanation, and mitigation, respectively. Section 7 discusses implications, and Section 8 concludes. 2 Related Work Multi-agent debate and self-reflection. Multi-agent debate improves LLM reasoning by having multiple instances challenge and refine each other’s answers 10 ; 16 ; 34 ; 3 , with applications ranging from mathematical reasoning to LLM-based evaluation 11 ; 2 . However, several studies report that debate does not consistently outperform single-agent baselines 24 ; 26 ; 30 ; 33 ; 31 , and 5 find that much of the observed benefit can be explained by aggregation alone. On the self-reflection side, while iterative refinement can improve outputs 19 , 13 show that without external feedback, LLMs often degrade their answers upon reconsideration. This finding directly motivates our self-reflection control: answer changes in debate cannot be attributed to peer influence without accounting for this baseline instability. Conformity and sycophancy in LLMs. Recent studies expose target models to peer stances and measure answer changes, drawing parallels to Asch’s conformity experiments 1 . These studies consistently find that peer exposure increases answer change rates 37 ; 29 ; 17 ; 7 ; 4 , with proposed mitigations including anonymization 6 , agent pairing strategies 32 , and dynamic prompting 21 . Related work on sycophancy in single-agent settings 23 ; 28 and unified signal-competition frameworks 35 treat conformity and sycophancy as related phenomena. However, these studies typically equate any answer change following peer exposure with conformity, without testing the counterfactual that the model would have maintained its answer absent peer input. Our self-reflection control provides this counterfactual, and our experimental design separates normative influence (stance-only) from informational influence (full reasoning), following the distinction introduced by 9 . Information quality and persuasion. Studies on adversarial attacks show that models can be misled by text mimicking valid reasoning while containing logical errors 25 ; 8 . Our information-gradient experiment provides a controlled measurement of this susceptibility in multi-agent interaction, comparing stance-only, invalid reasoning, and wrong reasoning conditions to disentangle the persuasive effects of reasoning form from reasoning content. 3 Methodology 3.1 Counterfactual Decomposition We study a MAD setting with N=5 agents. Each agent independently answers a multiple-choice question in Round 0, producing an initial answer a_{i}^{(0)} . In Round 1, agents on disagreement questions are exposed to peer information and asked to reconsider. We decompose answer changes into three sources using parallel counterfactual conditions. Let a_{i}^{(\text{sr})} , a_{i}^{(\text{so})} , and a_{i}^{(\text{r})} denote agent i ’s Round 1 answers under self-reflection, stance-only, and full reasoning conditions: • Spontaneous change: a_{i}^{(\text{sr})}\neq a_{i}^{(0)} . The agent changes upon re-examination with no peer information. • Stance-induced conformity: a_{i}^{(\text{sr})}=a_{i}^{(0)} , a_{i}^{(\text{so})}\neq a_{i}^{(0)} , and

Read the complete text