SOLVE-Med: Specialized Orchestration for Leading Vertical Experts across Medical Specialties

Roberta Di Marino, Giovanni Dioguardi, Antonio Romano, Giuseppe Riccio, Mariano Barone, Marco Postiglione, Flora Amato, Vincenzo Moscato · 2025

Read SOLVE-Med: Specialized Orchestration for Leading Vertical Experts across Medical Specialties by Roberta Di Marino, Giovanni Dioguardi, Antonio Romano, Giuseppe Riccio, Mariano Barone, Marco Postiglione, Flora Amato, Vincenzo Moscato (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-03542.txt.

Introduction

Roberta Di Marino and colleagues assemble SOLVE-Med, a team of small specialist models that answer medical questions together. Single models answering across all of medicine face hallucinations, bias, heavy compute, and privacy strain. SOLVE-Med routes each query to the right specialist among ten small models, each tuned on one medical field.

An orchestrator then blends the chosen specialists' answers into one reply. On Italian medical forum data across ten specialties, scores beat standalone models many times larger. All parts run locally, which protects patient privacy and cuts cost. The design trades one giant generalist for a panel that mirrors how real care splits by specialty.

Catalog record

CREATORRoberta Di Marino, Giovanni Dioguardi, Antonio Romano, Giuseppe Riccio, Mariano Barone, Marco Postiglione, Flora Amato, Vincenzo Moscato
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-03542

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SOLVE-Med: Specialized Orchestration for Leading Vertical Experts across Medical Specialties Roberta Di Marino, Giovanni Dioguardi, Antonio Romano, Giuseppe Riccio, Mariano Barone, Marco Postiglione, Flora Amato, Vincenzo Moscato Abstract Medical question answering systems face deployment challenges including hallucinations, bias, computational demands, privacy concerns, and the need for specialized expertise across diverse domains. Here, we present SOLVE-Med, a multi-agent architecture combining domain-specialized small language models for complex medical queries. The system employs a Router Agent for dynamic specialist selection, ten specialized models (1B parameters each) fine-tuned on specific medical domains, and an Orchestrator Agent that synthesizes responses. Evaluated on Italian medical forum data across ten specialties, SOLVE-Med achieves superior performance with ROUGE-1 of 0.301 and BERTScore F1 of 0.697, outperforming standalone models up to 14B parameters while enabling local deployment. Our code is publicly available on GitHub: https://github.com/PRAISELab-PicusLab/SOLVE-Med . 1 Introduction The rapid advancement of artificial intelligence in healthcare has catalyzed the emergence of sophisticated question-answering systems capable of addressing complex medical queries. With the Healthcare Chatbots Market projected to surpass a valuation of US$ 11.8 billion by 2033 8 8 8 https://www.globenewswire.com/news-release/2025/01/20/3011911/0/en/ Healthcare-Chatbots-Market , there is an urgent need for reliable, interpretable, and privacy-preserving medical AI systems that can operate effectively within clinical workflows. Large Language Models (LLMs) have demonstrated remarkable capabilities in medical question answering by enabling automatic access to complex knowledge across various domains [ 20 , 25 ] . However, their deployment in clinical contexts faces significant barriers, including hallucinations, bias, limited interpretability, high computational costs, and data privacy concerns associated with closed-source, cloud-based services [ 15 , 18 ] . Small Language Models (SLMs) represent a compelling alternative paradigm that addresses many of these challenges. By combining compact architectures with efficient fine-tuning on domain-specific data, SLMs require substantially fewer computational resources for both training and inference while enabling local deployment, reduced energy consumption, and enhanced privacy control [ 6 , 21 ] . Importantly, despite their reduced scale, well-tuned SLMs can deliver high task performance and generate accurate, context-aware responses even in complex medical scenarios [ 11 ] . This efficiency makes them particularly suitable for healthcare applications where resource constraints and data sensitivity are critical considerations. The multi-agent paradigm has gained considerable attention in question-answering systems, offering advantages in modularity, task separation, and system robustness [ 5 , 8 ] . In medical contexts, where queries often span multiple specialties and require diverse forms of expertise, this framework enables clearer functional boundaries and more reliable outputs [ 23 , 17 ] . However, existing approaches have not fully exploited the potential of combining specialized SLMs within a multi-agent architecture specifically designed for medical question answering. Here, we present SOLVE-Med (Specialized Orchestration for Leading Vertical Experts across Medical Specialties), a novel multi-agent system that addresses the specific demands of medical question answering through the strategic integration of domain-specialized SLMs. Our approach comprises three main components: (i) a Router Agent that performs multi-label classification to dynamically select relevant medical specialists based on query content, (ii) a pool of ten specialized SLMs, each fine-tuned on specific medical domains using Italian healthcare forum data gathered from Medicitalia 9 9 9 https://www.medicitalia.it/ and Dica33 10 10 10 https://www.dica33.it/ , and (iii) an Orchestrator Agent that synthesizes multiple expert responses into coherent, comprehensive answers. This architecture enables local deployment through compact models, significantly improving computational efficiency and preserving data privacy by eliminating dependence on external cloud infrastructure. SOLVE-Med Demo Features ✔ The demo features a multi-turn chat interface, allowing users to submit queries and engage in an interactive dialogue with the system. ✔ At each interaction step, SOLVE-Med presents not only the final response generated by the Orchestrator Agent but also the individual outputs from the selected medical specialists, along with the confidence scores assigned by the Router Agent. ✔ Users are also allowed to direct queries to specific agents within the pool of medical specialists, enabling targeted exploration of domain-specific expertise. Figure 1: Overview of the SOLVE-Med architecture and multi-turn interaction flow. Given a user question, the system first performs context-aware reformulation based on the conversation state. The reformulated query is then processed by the Router Agent, which assigns relevance scores to domain-specific specialist models (SLMs). The most relevant specialists are selected to generate candidate responses, which are then integrated by the Orchestrator Agent into a single coherent medical answer. The conversation state is continuously updated to support multi-turn interactions. 2 SOLVE-Med: Overview In this section, we present a detailed description of the SOLVE-Med architecture (see Figure 1 for an overview), which comprises three core components. First, a Router Agent (RA) dynamically selects the most suitable expert agents to address a given user query. Second, a pool of Medical Specialists , each consisting of a small language model pre-trained in a specific domain of medical expertise, provides domain-specific responses. Third, an Orchestrator Agent (OA) integrates the individual outputs from the specialists into a coherent and unified final response. Router Agent The Router Agent (RA) functions as a multi-label classifier that assigns one or more specialized medical agents to each input query. By routing questions to the most relevant specialists, the RA mimics the consultative nature of clinical workflows, enabling the system to provide multiple expert perspectives on complex queries. We implemented the RA by fine-tuning DistilBERT-base-multilingual-cased [ 19 ] , which supports rapid inference and low memory consumption, making it particularly suitable for deployment in resource-constrained medical environments. Training details are provided in Section 3.1 . Specialist selection is guided by the classifier’s multi-label output, using one of these two strategies: • Threshold-based Medical specialists are selected based on whether their associated label scores exceed a fixed confidence threshold. In our experiments, thresholds were chosen to maximize the F_{\beta} score, with \beta=2 and \beta=3 , thus prioritizing recall over precision during this selection phase. This emphasis on recall reflects the system’s design: the Orchestrator Agent subsequently refines the response by filtering out incorrect or irrelevant contributions, thereby enhancing overall precision. • Top-n-based Medical specialists corresponding to the top-n highest-scoring labels are chosen. We evaluated SOLVE-Med’s performance under top-2 and top-3 selection settings. Medical Specialists Once selected by the Router Agent, the medical specialists receive the user query and generate responses grounded in their respective areas of expertise. The specialist agents consist of a set of ten Small Language Models (SLMs), each fine-tuned on a distinct medical specialty using the LLaMA-3.2-1B-Instruct model [ 2 ] . To further enhance runtime efficiency, we employed a quantized version of the model, significantly reducing memory consumption and inference latency while preserving acceptable response quality. Each SLM was trained to generate coherent, domain-specific answers with consistent style and terminology. Full training details are provided in Section 3.1 . Orchestrator Agent The Orchestrator Agent (OA) is responsible for aggregating and synthesizing the outputs of the selected medical specialists. It receives both the user’s original question and the set of responses generated by the relevant SLMs, and produces a single, integrated answer that is coherent, medically sound, and comprehensive. The OA is implemented using a quantized version of the Gemma-2-9B-IT model [ 7 ] . Its larger parameter count w.r.t. medical specialists enables effective integration of their contributions, helping to mitigate issues such as omissions or oversimplification that

Read the complete text