ConvoGen: Enhancing Conversational AI with Synthetic Data: A Multi-Agent Approach

Reem Gody, Mahmoud Goudy, Ahmed Y. Tawfik · 2025

Read ConvoGen: Enhancing Conversational AI with Synthetic Data: A Multi-Agent Approach by Reem Gody, Mahmoud Goudy, Ahmed Y. Tawfik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17460.txt.

Introduction

Chatbots learn from conversations, and real conversations are scarce, private, and uneven. Reem Gody, Mahmoud Goudy, and Ahmed Tawfik present ConvoGen in 2025, a framework that manufactures training dialogues with a team of software agents. The system learns from a few examples, then keeps sampling fresh scenarios from a hub of examples that updates as it goes. That loop keeps the invented talks varied instead of repetitive.

The synthetic data can train or test conversational systems, or pad small datasets for jobs like sorting a speaker's intent or summarizing a chat. Experiments show the output is diverse and realistic enough to help. The authors offer the method as a way to build dialogue systems without harvesting private messages.

Catalog record

CREATORReem Gody, Mahmoud Goudy, Ahmed Y. Tawfik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17460

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ConvoGen: Enhancing Conversational AI with Synthetic Data: A Multi-Agent Approach Reem Gody, Mahmoud Goudy, Ahmed Y. Tawfik Abstract In this paper, we present ConvoGen: an innovative framework for generating synthetic conversational data using multi-agent systems. Our method leverages few-shot learning and introduces iterative sampling from a dynamically updated few-shot hub to create diverse and realistic conversational scenarios. The generated data has numerous applications, including training and evaluating conversational AI models, and augmenting existing datasets for tasks like conversational intent classification or conversation summarization. Our experiments demonstrate the effectiveness of this method in producing high-quality diverse synthetic conversational data, highlighting its potential to enhance the development and evaluation of conversational AI systems. I Introduction Recent advancements in conversational AI have significantly enhanced the integration of AI across various applications, thereby improving the overall user experience. These advancements span from virtual assistants powered by large language models (LLMs), to video conferencing applications that leverage LLMs to generate summaries and insights from meeting conversations, and messaging or email applications that utilize AI for rewriting, or generating smart responses. These applications can either involve interactions between an agent and a user, or they may encompass open-domain social multi-party conversations. Accordingly, open-domain conversational data is a crucial enabler for such applications, driving research into different approaches for their collection. One common approach is web crawling, exemplified by the DailyDialog dataset [ 1 ] , which was sourced from English learning websites. Another technique involves automatically creating conversations by scraping chat forums like Reddit [ 2 ] , [ 3 ] . Although scalable, this method often requires significant filtering to remove noise and toxic content. Crowdsourcing is another prevalent methodology for collecting conversational data. Zhang et al. [ 4 ] introduced the PERSONA-CHAT dataset by assigning personas to crowd workers, resulting in more diverse and engaging dialogues. Similarly, Rashkin et al. [ 5 ] collected conversations by having one annotator describe an emotion-triggering situation and another react to it. While crowdsourcing can yield natural conversations, it is often costly, time-consuming, and challenging to scale for large datasets or multi-party conversations. Existing multi-party conversation datasets are typically derived from scripted TV series [ 6 ] , movies [ 7 ] , or speech corpora like the AMI meeting corpus [ 8 ] . The generation of synthetic conversational data has become a focal point of research to augment existing datasets or create tailored data for specific scenarios [ 9 ] , [ 10 ] , [ 11 ] . Large Language Models (LLMs) have been extensively researched for their ability to generate synthetic conversational data for open-domain bots [ 12 ] , [ 13 ] , [ 14 ] , [ 15 ] and simulate human communication for applications like speech recognition [ 16 ] , [ 17 ] . A common strategy to diversify generated conversations is grounding them in common sense knowledge [ 13 ] , specific topics [ 14 ] , or personas [ 12 ] , [ 15 ] . For instance, Kim et al. [ 13 ] used common sense knowledge from a knowledge graph to generate diverse dyadic conversations with GPT-3.5, suggesting their approach could extend to multi-party conversations. Chen et al. [ 14 ] grounded conversations in topics and backgrounds, generating both dyadic and triadic dialogues. Lee et al. [ 12 ] and Jandaghi et al. [ 15 ] used personas to generate dyadic conversations, with the latter employing a generator-critic framework to ensure high-quality dialogues. Our research introduces ConvoGen 1 1 1 ConvoGen Link : an innovative multi-agent framework for generating diverse dyadic and multi-party conversations. Similar to previous works [ 12 ] , [ 15 ] , [ 16 ] , [ 17 ] , we utilize a persona-based approach to initiate group conversations between multiple agents. However, we employ an innovative approach to enhance the diversity of the generated conversations. Our primary contributions are as follows: 1. Multi-Agent Framework for Synthetic Conversation Generation: We propose a multi-agent framework as an intuitive solution to address the problem of generating multi-party conversational datasets involving two or more participants in an efficient and scalable way compared to traditional crowd-sourcing methods. The framework leverages LLM-generated experiences involving personas with specific relationships discussing a topic. By utilizing persona-based agents, we achieve diverse and varied conversations backed by the unique interactions between the matched personas that are likely to improve alongside advancements in persona generation [ 18 ] . 2. Iterative Sampling Technique: We introduce iterative sampling as a technique to enhance the LLM-based experience generator and increase the diversity in the conversations generated based on these experiences in the case of large-scale dataset generation. Our evaluation demonstrates that iterative sampling results in high lexical diversity (average Measure of Lexical Diversity (MTLD) score), validating its effectiveness in generating varied conversational content. 3. Extensive Evaluation: We conduct extensive evaluations of the datasets generated by our framework to assess the lexical diversity and groundedness of the generated conversations across various setups. Using the Measure of Lexical Diversity (MTLD) metric, we demonstrate that LLM-based agents produce conversations with higher lexical diversity compared to casual human conversational datasets. Additionally, our LLM-as-a-judge approach validates that the generated conversations are well-grounded in the input personas, and the situations in which they are involved. Fig. 1: The procedure for generating conversations using ConvoGen. First, the experience generator generates the personas, their relations, a situation, a topic and a conversation starter. Next, a group of agents initialized using these personas engage in a conversation using the generated conversation starter. II Methodology We develop a multi-agent based approach for general multi-party conversational data generation. In our approach, we use the AutoGen framework [ 19 ] to build our multi-agent system. We first give a brief overview of multi-agent systems and the AutoGen framework, and then we describe our approach for synthetic data generation. II-A Multi-agent Systems LLMs have demonstrated impressive capabilities in performing tasks, when configured with task-specific prompts, or assisted with access to external tools or data stores. In addition, it has been shown that LLMs are able to solve complex tasks when these tasks are broken down into simpler subtasks [ 20 ] . This has encouraged exploring the potential of building systems that involve multiple LLM agents, each specialized in doing a specific task, and allowing them to work together in a collaborative pattern to accomplish even more complex tasks. Accordingly, several frameworks have emerged to facilitate utilizing the multi-agent approach in building LLM applications such as CrewAI and AutoGen. These frameworks facilitate the integration and coordination of multiple LLM agents, enabling them to work seamlessly together to solve complex problems. II-B AutoGen Framework AutoGen is an open-source framework designed to build multi-agent conversational systems using LLMs. It supports asynchronous messaging and both event-driven and request-response interaction patterns. The framework implements advanced multi-agent design patterns, including reflection, handoffs, mixture of agents, multi-agent debate, and group chat. In the group chat pattern, agents communicate simultaneously, taking turns to publish messages, with a group chat manager maintaining the order of turns. The manager selects the next speaker using either a round-robin algorithm or an LLM-based mechanism, then broadcasts the selected speaker’s message to all agents. This renders group chat pattern

Read the complete text