Emergent Bias and Fairness in Multi-Agent Decision Systems

Maeve Madigan, Parameswaran Kamalaruban, Glenn Moynihan, Tom Kempton, David Sutton, Stuart Burrell · 2025

Read Emergent Bias and Fairness in Multi-Agent Decision Systems by Maeve Madigan, Parameswaran Kamalaruban, Glenn Moynihan, Tom Kempton, David Sutton, Stuart Burrell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-16433.txt.

Introduction

Maeve Madigan and five co-authors probe group AI for bias in their arXiv paper from 2025. Teams of AI agents deciding together can beat single models on prediction tasks. In high-stakes fields like consumer finance, though, biased group decisions mean regulatory breaches and real losses, and few methods exist to measure that risk.

Running large simulations across many multi-agent setups, with different ways for agents to communicate and collaborate, the team measured fairness on financial tabular tasks like credit scoring and income estimation. Bias emerged at the group level that could not be traced to any single agent's parts, evidence of genuinely collective behavior. The authors argue fairness risk in such systems is a real component of model risk. Multi-agent systems, they conclude, must be audited as whole entities, not piece by piece.

Catalog record

CREATORMaeve Madigan, Parameswaran Kamalaruban, Glenn Moynihan, Tom Kempton, David Sutton, Stuart Burrell
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-16433

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Emergent Bias and Fairness in Multi-Agent Decision Systems Maeve Madigan, Parameswaran Kamalaruban, Glenn Moynihan, Tom Kempton, David Sutton, Stuart Burrell Abstract Multi-agent systems have demonstrated the ability to improve performance on a variety of predictive tasks by leveraging collaborative decision making. However, the lack of effective evaluation methodologies has made it difficult to estimate the risk of bias, making deployment of such systems unsafe in high stakes domains such as consumer finance, where biased decisions can translate directly into regulatory breaches and financial loss. To address this challenge, we need to develop fairness evaluation methodologies for multi-agent predictive systems and measure the fairness characteristics of these systems in the financial tabular domain. Examining fairness metrics using large-scale simulations across diverse multi-agent configurations, with varying communication and collaboration mechanisms, we reveal patterns of emergent bias in financial decision-making that cannot be traced to individual agent components, indicating that multi-agent systems may exhibit genuinely collective behaviors. Our findings highlight that fairness risks in financial multi-agent systems represent a significant component of model risk, with tangible impacts on tasks such as credit scoring and income estimation. We advocate that multi-agent decision systems must be evaluated as holistic entities rather than through reductionist analyses of their constituent components. 1 Introduction Large language models (LLMs) have shown remarkable abilities across language-based tasks such as reasoning ( 49 ) and few-shot classification ( 7 ) . They have very quickly formed the basis of agents deployed to address real-world problems, increasingly often as part of multi-agent systems. As well as forming realistic simulations of human behavior ( 40 ; 39 ; 11 ) , multi-agent systems allow agents to collaborate on decision-making and reasoning tasks and reach higher performances than seen in individual LLMs ( 17 ) . This is most clearly seen in multi-agent debate, a method developed to enhance the performance and reasoning of agentic systems ( 15 ; 30 ; 54 ; 53 ; 16 ) . Agents are required to consider the decisions of other agents, to discuss their reasoning and to reach a consensus ( 26 ; 9 ) , which leads the multi-agent system to a more accurate final decision than that obtained by an individual agent. This exchange-of-thought ( 54 ) allows agents to improve based on external feedback, improving on techniques such as self-refinement ( 31 ; 50 ; 44 ) , chain-of-thought ( 49 ) and self-consistency ( 48 ) , all of which rely on the agent to improve its own response. Figure 1: Discussion paradigms for a multi-agent system of size N . (a) illustrates the Memory setup ( 54 ) that corresponds to a fully connected graph, where agents are provided with the output of all other agents and iteratively suggest refinements. (b) shows the Collective Refinement setup ( 26 ) , where each agent first generates a draft response independently, before refining their answer based on the drafts of all other N-1 agents. Generative AI is already transforming the financial sector ( 28 ; 14 ; 55 ; 18 ; 19 ) , powering applications from trading ( 55 ) to financial advisement ( 28 ) , and studies have explored its potential in forecasting credit risk ( 4 ; 41 ; 51 ) . Financial institutions are increasingly exploring multi-agent systems for applications such as financial question answering ( 18 ) and stock market prediction ( 19 ) . Despite their advantages, the deployment of multi-agent systems collides with a core challenge in the finance industry: model risk management ( 13 ) . Financial regulatory frameworks such as SR 11-7, SS 1/23 and the EU AI Act demand rigorous governance of models to prevent unintended bias or systemic risk. However, current approaches struggle to monitor agents due to their dynamic ability to plan, adapt to new information in their environment and to make autonomous decisions. This concern is amplified in multi-agent systems, where agents exchange and refine ideas in ways that can inadvertently form echo chambers, reinforcing existing biases. In high-stakes financial contexts such as credit scoring, this emergence of bias could have significant consequences for compliance. This is particularly true of model bias in a fairness context. LLMs are known to exhibit biases against protected characteristics such as race, age, and gender ( 20 ; 47 ) , inherited from stereotypes and prejudices present in their training data and observed in contexts such as simulations of political debates ( 46 ) and tabular classification ( 29 ) . Multi-agent systems may amplify biases inherent to LLMs ( 3 ; 6 ; 33 ; 12 ) . While prior work has examined the amplification of bias toward specific social conventions ( 3 ) or incorrect strategic reasoning ( 33 ) , their impact on fairness-related bias remains underexplored in the literature. Understanding how such bias emerges is particularly important for financial applications, where scarce labeled data makes LLMs attractive for few-shot classification ( 7 ; 43 ; 45 ; 24 ) and, at the same time, regulatory frameworks demand strict fairness compliance. Additionally, future multi-agent systems will likely involve agents controlled by different authorities and, without explicit control over bias in individual agents, we cannot prevent these biases from influencing the bias of the multi-agent system ( 1 ) . We perform a systematic study of bias in multi-agent systems through simulation experiments on financial decision-making tasks. We focus on credit scoring and income estimation as core applications where multi-agent systems could enhance decision-making through collaborative reasoning while operating on the structured, tabular data that dominates financial workflows. Our study finds that multi-agent systems exhibit unpredictable levels of bias on financial decision-making tasks relative to their LLM components. While some multi-agent systems are found to marginally reduce the bias relative to their constituent LLMs, many lead to significant and unpredictable increases in bias. Our simulations paint a clear picture: multi-agent systems must be independently evaluated for bias before their safe deployment in the financial domain; we cannot rely on the bias of their constituent LLMs to understand the system as a whole. 2 Related Work Tabular Classification with LLMs. LLMs are highly capable few-shot learners ( 7 ; 43 ) , and this ability has been extended to tabular data classification. TabLLM ( 24 ) demonstrates how serializing tabular data into natural language strings via template prompts enables LLMs to classify rows using in-context examples, often outperforming deep learning models. Subsequent benchmarks across multiple datasets ( 45 ) confirm the competitiveness of this approach in low-data regimes, including on the financial tabular datasets investigated in this work. However, the use of LLMs for tabular classification is not without its risks. Recent work has shown that this approach can lead to significant levels of bias, raising concerns about its use in sensitive applications ( 29 ) . Our work extends this study from individual LLMs to the dynamics of multiple interacting LLMs. Multi-Agent Debate. Multi-agent debate (MAD) has emerged as a powerful paradigm for complex problem-solving in LLMs. Core research in this area explores how multiple agents can collaborate to improve performance, as seen in foundational works ( 15 ; 30 ; 54 ; 53 ) . Notably, exchange-of-thought approaches demonstrate that MAD

Read the complete text