Chat-of-Thought: Collaborative Multi-Agent System for Generating Domain Specific Information
Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Dhaval Patel · 2025
Read Chat-of-Thought: Collaborative Multi-Agent System for Generating Domain Specific Information by Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Dhaval Patel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-10086.txt.
Introduction
Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, and Dhaval Patel build a team of AI agents that writes failure analysis documents together. Failure modes and effects analysis, or FMEA, is a standard safety worksheet for industrial gear. Drafting one by hand is slow and demands scarce expert time.
Their system, Chat-of-Thought, splits the work among language model agents with set roles. Tasks route themselves as the draft grows. A staged group discussion, with agents taking different personas, refines each table row by row. The workflow runs from templates and stays aware of context. Tests on equipment monitoring cases show the team can draft and check FMEA tables fit for expert review.
Catalog record
| CREATOR | Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Dhaval Patel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-10086
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Chat-of-Thought: Collaborative Multi-Agent System for Generating Domain Specific Information Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Dhaval Patel Abstract This paper presents a novel multi-agent system called Chat-of-Thought, designed to facilitate the generation of Failure Modes and Effects Analysis (FMEA) documents for industrial assets. Chat-of-Thought employs multiple collaborative Large Language Model (LLM)-based agents with specific roles, leveraging advanced AI techniques and dynamic task routing to optimize the generation and validation of FMEA tables. A key innovation in this system is the introduction of a Chat of Thought , where dynamic, multi-persona-driven discussions enable iterative refinement of content. This research explores the application domain of industrial equipment monitoring, highlights key challenges, and demonstrates the potential of Chat-of-Thought in addressing these challenges through interactive, template-driven workflows and context-aware agent collaboration. 1 Introduction In modern data-driven industries, teams often rely on diverse expertise to extract actionable insights and optimize operations. Collaborative efforts between subject matter experts, data scientists, and engineers play a critical role in analyzing complex systems, identifying risks, and improving decision-making. However, such processes are resource-intensive and require significant coordination. To address these challenges, we propose a novel framework, Chat-of-Thought, which leverages the role-playing capabilities of Large Language Models (LLMs) Achiam et al. 2023 , Touvron et al. 2023 , Jiang et al. 2023 to simulate collaborative, multi-agent environments Park et al. 2023 for domain-specific knowledge generation. The Chat-of-Thought system enables the creation of virtual personas, each representing a specific area of expertise. These personas operate collaboratively to generate hypothetical documents and insights, without requiring human intervention. By integrating inputs from diverse information sources—such as Failure Modes and Effects Analysis (FMEA), Key Performance Indicators (KPIs), diagnostic data, YAML configurations, synthetic datasets, and modeling code—this framework facilitates the exploration of complex scenarios in an automated and scalable manner. Figure 1: Overview of Chat-of-Thought. A key application of this framework is the automation of FMEA, a systematic reliability engineering method used to identify and mitigate potential failure modes in industrial systems. The manual creation of FMEA documents is labor-intensive and demands multidisciplinary expertise, making it an ideal candidate for automation. Our system, Chat-of-Thought, employs a multi-agent architecture with specialized roles to streamline FMEA generation, ensuring accuracy and scalability while significantly reducing the effort required. By combining the collaborative potential of simulated personas with diverse data inputs, the Chat-of-Thought system demonstrates a transformative approach to process optimization, risk identification, and decision-making. This work aims to illustrate the potential of LLMs to revolutionize traditional workflows, enabling industries to harness artificial intelligence for enhanced reliability and operational efficiency. Figure 2: Overview of Chat-of-Thought System Architecture 1.1 Application Domain and Problem Scenario The system is designed for industries managing complex and high-value assets, including pumps, boilers, chillers, and similar equipment. These assets are inherently prone to failures caused by mechanical, electrical, and operational issues, often resulting in significant downtime and costs. The manual creation of Failure Modes and Effects Analysis (FMEA) documents, a critical task in reliability engineering, suffers from limitations such as lack of scalability, inconsistencies, and delays in validation—particularly for out-of-scope (OOS) scenarios involving novel or less-common assets. The primary challenges in this domain include: • The requirement for highly specialized domain-specific expertise. • Accurate contextual understanding of diverse asset classes and their associated failure mechanisms. • Streamlined and efficient validation of FMEA data to minimize the workload of Subject Matter Experts (SMEs). 2 System Architecture and Key Components The proposed Chat-of-Thought system employs a collaborative multi-agent framework powered by Large Language Models (LLMs). Each agent within the system is designed with predefined roles, skills, and contextual system messages, enabling them to simulate domain-specific expertise. The architecture is structured into the following key stages: 2.1 Initialization of Assistant Agents Agents are initialized with predefined attributes that simulate specific roles essential for FMEA generation. These roles include, but are not limited to: • Facilitator: Oversees the overall interaction and ensures task progression. • Reliability Engineer: Focuses on identifying potential failure modes and their underlying mechanisms. • Quality Engineer: Reviews generated insights for accuracy and consistency. • SME Validator: Validates outputs to ensure they meet domain-specific standards. • Summarizer: Summarizes the document and answers into a concise and readable format. Each agent A_{i} is represented as a tuple (R_{i},S_{i},M_{i}) , where R_{i} denotes the role, S_{i} represents the skill set, and M_{i} specifies the system messages guiding the agent’s behavior. 2.2 Context Discovery The system begins by identifying the asset class (e.g., Pump - Vertical Close-Coupled) and its operational parameters. Relevant failure modes, their causes, and associated effects are dynamically extracted from domain-specific knowledge repositories and historical data. This stage ensures that the analysis is tailored to the specific context of the asset being examined. Figure 3: Round progression. Chat-of-Thought is a flexible framework where agents collaborate using various methods. 2.3 Multi-Round Chain of Interactions The core innovation of Chat-of-Thought lies in its Chat-of-Thought mechanism, where agents engage in iterative, multi-persona-driven discussions to collaboratively refine outputs. This approach extends beyond the traditional Chain-of-Thought framework by fostering dynamic, context-aware dialogue across specialized personas. The process consists of the following steps: 1. Question Selection: The Facilitator Agent selects a question Q from a predefined question bank Q_{B} . 2. Dynamic Persona Assignment: The Question Assignment Agent routes the selected question to the most suitable persona A_{i} based on contextual understanding C and skill alignment S_{i} . 3. Response Generation: The assigned persona generates a response R , incorporating its domain-specific expertise and LLM capabilities. 4. Summary Aggregation: The Question Summary Agent synthesizes responses into a cohesive and actionable summary for downstream processing. 2.4 Template-Driven Routing To ensure high-quality outputs, the system employs a template-based routing mechanism. Each template T defines structured guidelines for routing questions to the appropriate personas. These templates are dynamically adjusted to maintain contextual relevance and optimize the quality of generated FMEA data. By leveraging this template-driven approach, Chat-of-Thought ensures consistency across interactions while adapting to the nuances of various scenarios. This architecture, with its emphasis on collaborative intelligence and role-specific expertise, empowers industries to automate labor-intensive FMEA tasks effectively while maintaining precision and scalability. 2.5 Quality check At the end of each round there is a quality control to filter out useless questions using a pre-trained classifier, and remove duplicates questions/answers using a self-BLEU score Papineni et al. 2002 threshold. 3 AI Techniques and Innovations Chat-of-Thought integrates advanced AI techniques to streamline FMEA generation and enhance performance in complex tasks. At its core, the system leverages the power of Large Language Models (LLMs) for domain-specific reasoning, natural language understanding, and generating high-quality outputs. Task assignments are guided by context-aware routing mechanisms, where internal-thought templates align agent skills with the query’s specific requirements. Multi-agent collaboration ensures iterative refinement of outputs, while a stateful design allows the agents to manage multi-step and context-dependent processes efficiently. The system operates through a multi-phase process that combines failure model analysis, iterative learning, and question-answering frameworks. The initial phase ( Round 1 ) evaluates zero-shot learning capabilities, testing the system’s baseline performance without task-specific examples. This is followed by in-context learning ( Round 2 ), where scenarios derived from Failure Modes and Effects Analysis (FMEA) guide response generation and incorporate human feedback for refinement. Subsequent rounds build on this foundation ( Round