AR-Med: Automated Relevance Enhancement in Medical Search via LLM-Driven Information Augmentation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Conference: ; ;
Chuyue Wang 1 , Jie Feng 2 , Yuxi Wu 3 , Hang Zhang 1 , Zhiguo Fan 1 , Bing Cheng 1 , Wei Lin 1 1 Meituan Inc., Beijing, China 2 Tsinghua University, Beijing, China 3 Independent Researcher, Beijing, China [email protected] · 2025
Read AR-Med: Automated Relevance Enhancement in Medical Search via LLM-Driven Information Augmentation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Conference: ; ; by Chuyue Wang 1 , Jie Feng 2 , Yuxi Wu 3 , Hang Zhang 1 , Zhiguo Fan 1 , Bing Cheng 1 , Wei Lin 1 1 Meituan Inc., Beijing, China 2 Tsinghua University, Beijing, China 3 Independent Researcher, Beijing, China [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03737.txt.
Introduction
Searching a healthcare platform is a high-stakes task, where a poor match can point a patient toward the wrong care. Chuyue Wang and colleagues at Meituan and Tsinghua University present AR-Med in a 2025 preprint, a system already running on an online medical delivery platform. Ordinary search methods often miss the meaning of complicated, detailed medical queries.
AR-Med uses a large language model whose reasoning is grounded in verified medical knowledge fetched at query time, reducing made-up answers. To keep the service fast and affordable, the team distils big teacher models into compact student models. They also built LocalQSMed, a benchmark labelled by several experts, to keep offline scores honest against online behaviour. In experiments the system reached accuracy above ninety-three percent, a large jump over the platform's previous system, with matching gains in user satisfaction.
Catalog record
| CREATOR | Chuyue Wang 1 , Jie Feng 2 , Yuxi Wu 3 , Hang Zhang 1 , Zhiguo Fan 1 , Bing Cheng 1 , Wei Lin 1 1 Meituan Inc., Beijing, China 2 Tsinghua University, Beijing, China 3 Independent Researcher, Beijing, China [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AR-Med: Automated Relevance Enhancement in Medical Search via LLM-Driven Information Augmentation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Conference: ; ; Chuyue Wang 1 , Jie Feng 2 , Yuxi Wu 3 , Hang Zhang 1 , Zhiguo Fan 1 , Bing Cheng 1 , Wei Lin 1 1 Meituan Inc., Beijing, China 2 Tsinghua University, Beijing, China 3 Independent Researcher, Beijing, China [email protected] Abstract . Accurate and reliable search on online healthcare platforms is critical for user safety and service efficacy. Traditional methods, however, often fail to comprehend complex and nuanced user queries, limiting their effectiveness. Large language models (LLMs) present a promising solution, offering powerful semantic understanding to bridge this gap. Despite their potential, deploying LLMs in this high-stakes domain is fraught with challenges, including factual hallucinations, specialized knowledge gaps, and high operational costs. To overcome these barriers, we introduce AR-Med , a novel framework for A utomated R elevance assessment for Med ical search that has been successfully deployed at scale on the Online Medical Delivery Platforms. AR-Med grounds LLM reasoning in verified medical knowledge through a retrieval-augmented approach, ensuring high accuracy and reliability. To enable efficient online service, we design a practical knowledge distillation scheme that compresses large teacher models into compact yet powerful student models. We also introduce LocalQSMed, a multi-expert annotated benchmark developed to guide model iteration and ensure strong alignment between offline and online performance. Extensive experiments show AR-Med achieves an offline accuracy of over 93%, a 24% absolute improvement over the original online system, and delivers significant gains in online relevance and user satisfaction. Our work presents a practical and scalable blueprint for developing trustworthy, LLM-powered systems in real-world healthcare applications. 1. Introduction In the realm of healthcare, accurate pharmaceutical search and recommendation are paramount, as they directly impact public health and safety. With the proliferation of online platforms, medications and medical products are readily available for purchase anytime and anywhere, heightening the need for precise relevance matching to prevent misuse, adverse effects, or ineffective treatments. Traditional search and recommendation methods, primarily based on small-scale deep learning models, face significant limitations in pharmaceutical domains. These methods include collaborative filtering (e.g., Slope One, user/item similarity ( Su and Khoshgoftaar 2009 ; Shao and Xie 2020 ) ), content-based filtering (e.g., keyword matching, semantic analysis ( Manning et al. 2008 ) ), and hybrid approaches like knowledge graph-based systems ( Guo et al. 2020 ) . They struggle with: (1) limited comprehension, failing to handle ambiguous queries like "wind-cold common cold" or "001" ( Chen et al. 2022 ) ; (2) reliance on costly expert-curated knowledge, increasing maintenance costs ( Chowdhury 2010 ) ; and (3) slow adaptation to new drugs, regulations, and user demands, leading to outdated relevance ( Ricci et al. 2010 ) . These challenges highlight the need for more robust solutions. Large language models (LLMs), on the other hand, hold immense potential to revolutionize this field through their advanced natural language understanding, commonsense reasoning, and zero/few-shot learning abilities ( Touvron et al. 2023 ; OpenAI 2022 ; Wei et al. 2022a ) . These capabilities enable LLMs to delve deeper into user query intents and the intrinsic logic of medical texts, thereby substantially enhancing the quality of relevance assessments. For instance, in the domain of natural language understanding, models like GPT-4 have demonstrated expert-level performance on medical question-answering tasks, such as diagnosing complex cases or interpreting clinical queries, by leveraging vast pre-trained knowledge to achieve accuracies comparable to human specialists on benchmarks like USMLE ( Nori et al. 2023 ) . In commonsense reasoning, techniques such as chain-of-thought prompting allow LLMs to break down multi-step problems, improving arithmetic, symbolic reasoning, and logical inference in diverse scenarios, as shown in studies on PaLM models ( Wei et al. 2022b ) . Zero/few-shot learning further empowers LLMs to generalize to unseen tasks with minimal examples; for example, models like GPT-3 excel in translation, code generation, and multitask adaptation without task-specific training ( Radford et al. 2019 ) , while specialized variants like CodeGen handle program synthesis in coding domains ( Nijkamp et al. 2023 ) . Across various fields ( Zhang et al. 2024 ; Wang et al. 2023 ; Scao et al. 2022 ; Feng et al. 2025b ; Feng et al. 2025a ; Lan et al. 2025b ; Lan et al. 2025a ) , LLMs have shown versatility: in e-commerce search, they enhance relevance judgments for product queries ( Soviero et al. 2024 ) ; in biomedical applications, models like HuaTuo and BiomedGPT fine-tuned on Chinese medical knowledge enable accurate diagnosis and knowledge retrieval ( Wang et al. 2023 ; Zhang et al. 2024 ) ; and in tool-augmented settings, Toolformer self-teaches API usage for tasks like calculation or search integration ( Schick et al. 2023 ) . These examples illustrate how LLMs’ emergent abilities, such as those in BLOOM’s multilingual processing ( Scao et al. 2022 ) or Llama 3’s broad cognitive tasks ( Grattafiori et al. 2024 ) , drive innovations in knowledge-intensive applications. Figure 1 . An illustration of the proposed framework, encompassing three key components: Knowledge Retrieval, Expert Rule Guidance, and Cross-modal Information Matching. However, LLMs are not without drawbacks: they often lack domain-specific professional knowledge, are prone to severe hallucinations that can lead to erroneous medical advice, and impose high application costs in terms of latency and computational resources, particularly in high-throughput industrial scenarios like daily billions of search requests on platforms such as the Online Medical Delivery Platforms, a leading local services provider. For example, despite strong general capabilities, LLMs like GPT-4 may produce inaccurate outputs in specialized medical contexts due to insufficient domain grounding, as evidenced in evaluations showing persistent errors in clinical reasoning ( Liévin et al. 2023 ) . Hallucinations—fabricating plausible but false information—remain a critical issue, with surveys highlighting risks in high-stakes domains like healthcare ( Zhao et al. 2025 ; Zhou et al. 2023 ) . Moreover, the enormous scale of models such as PaLM 2 (up to 540B parameters) ( Chowdhery and Narang 2023 ) or GPT-NeoX-20B ( Black et al. 2022 ) demands massive compute resources for training and inference, leading to inefficiencies in deployment, as noted in compute-optimal analyses ( Hoffmann et al. 2022 ) and efforts to mitigate via optimizations like DeepSpeed ( Rasley et al. 2020 ) . These limitations underscore the need for hybrid approaches, such as retrieval-augmented generation (RAG) ( Lewis et al. 2020 ) , to complement LLMs in practical settings. To harness the semantic understanding and reasoning strengths of LLMs while mitigating their weaknesses—namely, insufficient specialized knowledge and hallucination risks—we propose a comprehensive LLM-based framework tailored for medical search and recommendation relevance. Building on the Online Medical Delivery Platforms’s diverse and complex real-world scenarios, our approach addresses these challenges through three key innovations. First, we construct a retrieval-augmented framework that deeply integrates LLMs’ reasoning capabilities with external, trustworthy medical knowledge bases. This framework employs precise knowledge retrieval, expert-guided rules, and cross-modal information matching to furnish LLMs with verified contextual information for decision-making, thereby anchoring outputs to ensure professionalism, accuracy, and reduced risks in medical