MATCHA: Can Multi-Agent Collaboration Build a Trustworthy Conversational Recommender?

Zheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao · 2025

Read MATCHA: Can Multi-Agent Collaboration Build a Trustworthy Conversational Recommender? by Zheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20094.txt.

Introduction

Hui and colleagues present MATCHA in twenty twenty-five, a chat based system that suggests video games to players. Movie tools can rely on fixed lists and simple tastes. Games change fast, and taste depends on skill, play style, and gear. Open chat also brings a risk of unsafe replies.

MATCHA splits the work among a team of small agents that each do one job. Some read what the user wants, some fetch games, and some rank choices and check for risk. Tests on real user requests beat six older methods on eight scores. The system also cut bias toward hit games and blocked most trick prompts.

Catalog record

CREATORZheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20094

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MATCHA: Can Multi-Agent Collaboration Build a Trustworthy Conversational Recommender? Zheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao Abstract In this paper, we propose a multi-agent collaboration framework called MATCHA for conversational recommendation system, leveraging large language models (LLMs) to enhance personalization and user engagement. Users can request recommendations via free-form text and receive curated lists aligned with their interests, preferences, and constraints. Our system introduces specialized agents for intent analysis, candidate generation, ranking, re-ranking, explainability, and safeguards. These agents collaboratively improve recommendations accuracy, diversity, and safety. On eight metrics, our model achieves superior or comparable performance to the current state-of-the-art. Through comparisons with six baseline models, our approach addresses key challenges in conversational recommendation systems for game recommendations, including: (1) handling complex, user-specific requests, (2) enhancing personalization through multi-agent collaboration, (3) empirical evaluation and deployment, and (4) ensuring safe and trustworthy interactions. 1 Introduction Figure 1: Comparison of traditional (a) and conversational (b, c) game recommendation systems. Discovering new games among millions of options spanning genres and platforms is often challenging. A conversational recommendation system (CRS) Christakopoulou et al. 2016 ; Lei et al. 2020 ; Jannach et al. 2021 ; Friedman et al. 2023 ; Fang et al. 2024 simplifies this process by allowing users to describe their preferences in natural language, as illustrated in Figure 1 . For example, users might ask, "I want a multiplayer adventure game with minimal violence" or "Suggest games I can play with my younger siblings on a tablet." Such requests require a system capable of understanding user intent, context, and constraints. While CRS systems have proven effective in domains such as movies, music, and products He et al. 2023 ; Fang et al. 2024 , practical implementations for game recommendation remain rare. LLM-based CRS face three key challenges: (1) reliance on static knowledge bases Feng et al. 2023 ; Dai et al. 2024 , which prevents real-time updates and limits recommendations to pre-existing training data; (2) balance personalization and accuracy, as recommendation systems often prioritize well-known items over niche or newly released titles He et al. 2023 ; eun Yoon et al. 2024 ; Wang et al. 2025 ; and (3) the lack of trustworthiness, safeguards, and explainability, which are critical for ensuring safety and trust in dynamic, user-interactive systems. We argue that effective CRS systems require improved agent reasoning, better collaboration, safety measures, and clear, user-focused recommendation explanations. To address these gaps, we propose MATCHA , which stands for Multi-Agent System Collaboration for Trustworthy Conversational Recommendations. In this system, specialized agents are assigned distinct roles within the recommendation pipeline, including intent analysis, candidate generation, ranking, re-ranking, and explainability. By distributing tasks among agents, the framework can parse complex user queries, generate diverse game candidates, facilitate reasoning through multi-LLM collaboration, and deliver personalized recommendations with detailed explanations. A dedicated jailbreak prevention agent ensures the system appropriately handles inappropriate or unanswerable queries, thereby enhancing safety and ethical standards. Additionally, the decoupled agent design allows for the independent selection of backbone models for each agent, providing flexibility to balance cost and performance. For example, LLaMA3.1 Dubey et al. 2024 exhibits superior jailbreak prevention capabilities compared to GPT-4o, while GPT-4o Mini offers a more cost-effective alternative. Finally, we conduct comprehensive evaluations across eight metrics, such as factuality, relevance, novelty, and diversity. Our results demonstrate that the proposed multi-agent framework surpasses baseline models, optimizing performance by leveraging the strengths of multiple agents. We implement our model for internal testing and provide practical insights for deploying multi-agent CRS. Our contributions can be summarized in threefold: • We propose a novel multi-agent architecture that coordinates specialized agents for game recommendation. • We conduct extensive evaluations using real user interactions, demonstrating that our multi-agent approach achieves SOTA with higher accuracy, diversity, and user satisfaction compared to single-agent baselines. • We provide insights from implementing and testing our system, highlighting key considerations for deploying multi-agent CRS in real-world settings. 2 Related Work 2.1 Conversational Recommendation System The field of conversational recommendation systems (CRS) has garnered significant attention in recent years due to its potential to enhance user interaction and deliver personalized recommendations. Early works in CRS primarily relied on rule-based and retrieval-based methods Sarwar et al. 2001 ; Cheng et al. 2016 . Recent advancements in large language modelS Brown 2020 have greatly enhanced the capabilities of CRS, enabling systems to process and respond to complex natural language inputs with improved contextual understanding. LLMs have set new benchmarks in natural language processing, making them particularly well-suited for conversational tasks. Studies such as Lei et al. 2020 , Wang et al. 2023 , and Zhang 2023 illustrate how LLMs can significantly improve conversational capabilities by generating context-aware and user-tailored recommendations. Approaches such as MACRS Fang et al. 2024 leverage multi-agent systems, while others Friedman et al. 2023 ; Li et al. 2024 incorporate supplementary tools. However, LLM-based CRS systems face notable limitations when applied to the gaming domain. Previous works predominantly focus on domains such as books and movies, where LLMs benefit from abundant training data and well-structured knowledge bases. 2.2 Risk factor in LLM-based Dialogue System LLM-based dialogue systems Yi et al. 2024b present significant advancements in natural language understanding and generation, offering transformative capabilities for conversational applications. However, their deployment introduces several critical risk factors that must be addressed to ensure safety, reliability, and user trust Hui et al. 2024a . A primary concern is the vulnerability of LLMs to adversarial queries, which can exploit their generative capabilities to produce harmful or inappropriate outputs Cao et al. 2024 ; Liu et al. 2023 ; Yi et al. 2024a ; Chiang and Lee 2023 ; Hui et al. 2024b . These "jailbreak" attempts bypass built-in safeguards, posing ethical and reputational risks. Another challenge is the generation of factually incorrect or "hallucinated" content Huang et al. , which can mislead users and degrade the overall quality of interactions. 3 Problem Formulation Figure 2: Overview of Our MATCHA framework. The system processes user requests through safety agents, generates game candidates using diverse tools, refines them via ranking and reflection agents, and provides final recommendations with detailed explanations. Let the conversational recommendation system (CRS) be modeled as a multi-agent framework \mathcal{M}=\{\mathcal{A}_{1},\mathcal{A}_{2},\dots,\mathcal{A}_{n}\} , where each agent \mathcal{A}_{i} specializes in specific subtasks such as intent understanding, candidate generation, or explanation generation. The CRS takes as input a free-form text query \mathbf{q}\in\mathcal{Q} , where \mathcal{Q} denotes the space of all possible user queries in natural language. The goal of the CRS is to produce k recommended experiences \{r_{1},r_{2},\dots,r_{k}\}\subset\mathcal{R} and corresponding explanations \{e_{1},e_{2},\dots,e_{k}\}\subset\mathcal{E} , where \mathcal{R} is the space of all candidate recommendations and \mathcal{E} is the space of explanation texts. Formally, the problem can be defined as: f_{\text{CRS}}:\mathcal{Q}\to\mathcal{R}^{k}\times\mathcal{E}^{k} where f_{\text{CRS}} is the mapping function implemented by the CRS. For a given user query \mathbf{q} , the system aims to maximize the relevance of the recommended experiences r_{i} and the quality of the explanations e_{i} . This can be expressed as an optimization problem: \max_{\{r_{1},\dots,r_{k}\},\{e_{1},\dots,e_{k}\}}\sum_{i=1}^{k}\big(\text{Rel}(r_{i},\mathbf{q})+\alpha\cdot\text{Qual}(e_{i},r_{i},\mathbf{q})\big) where: • \text{Rel}(r_{i},\mathbf{q}) measures the relevance of the recommendation r_{i} to the query \mathbf{q} , • \text{Qual}(e_{i},r_{i},\mathbf{q}) evaluates the quality of the explanation e_{i} in justifying the recommendation r_{i} with respect to \mathbf{q} , • \alpha>0 is a hyperparameter that balances the

Read the complete text