Towards Cognitive Synergy in LLM-Based Multi-Agent Systems: Integrating Theory of Mind and Critical Evaluation
Adam Kostka ([email protected]), Jarosław A. Chudziak ([email protected]) · 2025
Read Towards Cognitive Synergy in LLM-Based Multi-Agent Systems: Integrating Theory of Mind and Critical Evaluation by Adam Kostka ([email protected]), Jarosław A. Chudziak ([email protected]) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21969.txt.
Introduction
Adam Kostka and Jarosław A. Chudziak study how teams of language model agents could reason together better, in a paper from 2025. Single agents do well alone but falter at the higher-order thinking that teamwork needs. Human teams gain synergy by modeling each other's minds and by structured critique that exposes gaps and bias.
The authors fold both skills into a multi-agent framework: agents track what partners likely know and grade each other's reasoning in set rounds. A case study on complex decisions showed more coherent and adaptive joint work. The paper frames the design as a step toward collective intelligence beyond the sum of parts.
Catalog record
| CREATOR | Adam Kostka ([email protected]), Jarosław A. Chudziak ([email protected]) |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-21969
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Cognitive Synergy in LLM-Based Multi-Agent Systems: Integrating Theory of Mind and Critical Evaluation Adam Kostka ([email protected]), Jarosław A. Chudziak ([email protected]) Abstract Recently, the field of Multi-Agent Systems (MAS) has gained popularity as researchers are trying to develop artificial intelligence capable of efficient collective reasoning. Agents based on Large Language Models (LLMs) perform well in isolated tasks, yet struggle with higher-order cognition required for adaptive collaboration. Human teams achieve synergy not only through knowledge sharing, but also through recursive reasoning, structured critique, and the ability to infer others’ mental states. Current artificial systems lack these essential mechanisms, limiting their ability to engage in sophisticated collective reasoning. This work explores cognitive processes that enable effective collaboration, focusing on adaptive theory of mind (ToM) and systematic critical evaluation. We investigate three key questions. First, how does the ability to model others’ perspectives enhance coordination and reduce redundant reasoning? Second, to what extent does structured critique improve reasoning quality by identifying logical gaps and mitigating biases? Third, the interplay of these mechanisms can lead to emergent cognitive synergy, where the collective intelligence of the system exceeds the sum of its parts. Through an empirical case study on complex decision making, we show that the integration of these cognitive mechanisms leads to more coherent, adaptive, and rigorous agent interactions. This article contributes to the field of cognitive science and AI research by presenting a structured framework that emulates human-like collaborative reasoning MAS. It highlights the significance of dynamic ToM and critical evaluation in advancing multi-agent systems’ ability to tackle complex, real-world challenges. Keywords: Multi-Agent Systems; Theory of Mind; Critical Evaluation; Cognitive Synergy; Introduction Rapid progress in large language models (LLMs) has allowed human-level performance in some activities of natural language, including translation, summarization, and problem-solving Naveed et al. (2024) . However, LLMs continue to face challenges in multi-agent collaborative environments involving effective coordination, rigorous evaluation, and adaptive reasoning Qiu et al. (2024) . Unlike human teams, which achieve cognitive synergy by integrating diverse knowledge, iterative critique, and dynamic mutual understanding (Theory of Mind), LLM-based systems often lack mechanisms for structured, adaptive coordination, risking fragmented solutions. Figure 1: Conceptual Cycle of Cognitive Synergy in Multi-Agent Systems Theory of Mind (ToM) and structured critique are two significant mechanisms that can enable cognitive synergy in human collaboration. ToM allows individuals to infer others’ beliefs, intentions, and knowledge states Premack & Woodruff (1978) ; Wellman et al. (1990) , facilitating effective coordination and redundancy reduction Dechurch & Mesmer-Magnus (2010) . Structured critique, within this context, refers to a systematic process where designated agents or internal mechanisms evaluate the reasoning, evidence, and potential biases within the arguments presented by other agents, aiming to improve rigor and identify logical weaknesses, analogous to processes like peer review and argumentation in human reasoning Mercier & Sperber (2011) ; Mercier (2016) . Despite playing a central role in human cooperative endeavors, such mechanisms are hugely underrepresented in today’s multiagent AI systems, thus limiting their ability for sophisticated cooperative reasoning. Most modern LLM-based Multi-Agent Systems (MAS) also tend to rely on static role assignments and task divisions, not representing adaptively the shifting perspectives of their counterparts. Recent approaches that incorporate LLMs have improved interactions based on language Schmidgall et al. (2025) , yet, such approaches frequently fail to achieve genuine coordination, since agents do not infer each other’s reasoning states or recursively refine their arguments through systematic criticism P. Li et al. (2024) . This work presents a MAS framework implementing cognitive synergy via dynamic ToM and structured critique. Figure 1 provides a conceptual overview of the interacting cognitive functions that can lead to synergy in MAS. Our specific approach features expert agents, a Critic, an Orchestrator, and an Integrator using a knowledge base and Clingo solver. The goal is synergy through integrated contributions, critique, and adaptive coordination. A case study on strategic business decision making investigates: 1) How dynamic ToM affects coordination and redundancy. 2) How a Critic Agent improves reasoning quality and bias mitigation. 3) Whether these mechanisms yield emergent cognitive synergy, exceeding the sum of parts. This work advances MAS design by combining cognitive science principles with AI concepts to present a framework intended to instantiate key aspects of collaborative human reasoning. The results underscore specialization, iterative critique, and adaptive coordination in AI systems for sophisticated decision-making. Related Work Collaborative AI system architectures have long been motivated by human cognition, and MAS research has long focused on task decomposition and pre-coordinated strategies to coordination Wooldridge (2009) ; Jennings et al. (1998) . Although such systems have been effective for structured workflows, they do not meet the flexible and adaptive collaboration of human teams. More recent attempts to integrate LLMs into MAS frameworks have improved language comprehension and communication X. Li et al. (2024) ; Muthusamy et al. (2023) , yet much work remains to be done to improve collaborative intelligence. One of the most serious limitations of current LLM-Based Multi-Agent Systems is the poor development of Theory of Mind (ToM), the ability to represent such things as knowledge states, intentions, and cognitive processes of other agents Premack & Woodruff (1978) ; Wellman et al. (1990) . In human collaboration, ToM often helps parties anticipate others’ contributions, align views, and avoid duplicating effort. Developmental psychology has shown that ToM is crucial in collaboration and social learning Tomasello et al. (2005) . In multi-agent systems, dynamic ToM could enable iteratively improved responses, improving coordination and efficiency between agents Kostka & Chudziak (2024) . Another part of fruitful collaboration is critical evaluation. Structured critique brings much improvement to human decision making, such as through peer review and deliberative reasoning processes Mercier (2016) . Most existing designs of MAS do not have such formally implemented mechanisms of critique, which could help reduce fallacious reasoning and weak conclusions. Some new frameworks offer self-correction for LLM-based systems Pan et al. (2023) , but very few include an autonomous critic agent to perform structured critique and systematically assess the robustness of reasoning. Mixture-of-Agents (MoA) models Wang et al. (2024) compose agents using large language models in a hierarchy, where each agent improves its responses by learning from past interactions. This framework supports dynamic answers, and also helps achieve better performance on very complex reasoning tasks. MoA mostly enhances the overall reasoning ability of different agents and focuses less on inter-agent evaluation or Theory of Mind H. Li et al. (2023) . Sparse Mixture-of-Agents (SMoA) D. Li et al. (2024) builds on MoA. SMoA optimizes efficiency via selective communication methods (like Response Selection, Early Stopping) to minimize interactions between agents, thus reducing costs while preserving view diversity. Like MoA, however, SMoA primarily centers on optimizing agents and workload allocation, offering no deeper integration of structured criticism or ToM within these systems Gou et al. (2024) ; Cross et al. (2024) . Higher-level cognitive mechanisms could, therefore, further enhance collaborative AI reasoning. Figure 2: Illustrative Comparison of Hypothesized Agent Response Patterns With and Without Theory of Mind (ToM) Capabilities. Our work extends these traditions by formally including cognitive mechanisms in the design of MAS. In contrast to previous work that assumed agent interactions to be static or task-specific, or focused on supporting human collaboration in specific domains like knowledge discovery