PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering
Yiqing Zhang, Xiaozhong Liu, Fabricio Murai · 2026
Read PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering by Yiqing Zhang, Xiaozhong Liu, Fabricio Murai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27335.txt.
Introduction
Zhang and co-authors present PubMed Reasoner, a biomedical question answering agent that grounds answers in retrieved evidence. A self-critic first refines the query by checking MeSH term coverage and redundancy. Reflective retrieval then reads article batches until the evidence suffices. Final answers carry explicit citations to the gathered sources. With a GPT-4o backbone, the system scored 78.32 percent on PubMedQA. That slightly surpassed human expert performance on the benchmark. Judge evaluations also preferred its answers for grounding and trustworthiness. The design controls compute and token costs throughout.
Catalog record
| CREATOR | Yiqing Zhang, Xiaozhong Liu, Fabricio Murai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72127 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27335
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering Yiqing Zhang, Xiaozhong Liu, Fabricio Murai Abstract Trustworthy biomedical question answering (QA) systems must not only provide accurate answers but also justify them with current, verifiable evidence. Retrieval-augmented approaches partially address this gap but lack mechanisms to iteratively refine poor queries, whereas self-reflection methods kick in only after full retrieval is completed. In this context, we introduce PubMed Reasoner, a biomedical QA agent composed of three stages: self-critic query refinement evaluates MeSH terms for coverage, alignment, and redundancy to enhance PubMed queries based on partial (metadata) retrieval; reflective retrieval processes articles in batches until sufficient evidence is gathered; and evidence-grounded response generation produces answers with explicit citations. PubMed Reasoner with a GPT-4o backbone achieves 78.32% accuracy on PubMedQA, slightly surpassing human experts, and showing consistent gains on MMLU Clinical Knowledge. Moreover, LLM-as-judge evaluations prefer our responses across: reasoning soundness, evidence grounding, clinical relevance, and trustworthiness. By orchestrating retrieval-first reasoning over authoritative sources, our approach provides practical assistance to clinicians and biomedical researchers while controlling compute and token costs. 1 Introduction Figure 1: RAG, Self-reflection vs. PubMed Reasoner. (a) RAG baselines: uses few-shot exemplars or custom databases but lack retrieval feedback. (b) Self-reflection agents: inspired our proposal; generates responses first and only reflects after completion. (c) PubMed Reasoner: a search-first approach that performs self-critic query refinement, reflective article retrieval in batches w/ early stopping once evidence is sufficient, and evidence-grounded response generation with explicit citations. Trustworthiness is essential in biomedical domains. Biomedical question answering (QA) systems must be factually grounded, current, and interpretable. Yet, large language models (LLMs) that rely primarily on parametric memory can hallucinate 13 , drift out of date, or omit key evidence 10 ; 26 . While prior works have explored retrieval-augmented methods, to our knowledge, this is the first work to explicitly support citation-backed responses on biomedical QA datasets, ensuring that each response is transparently grounded in published evidence. Existing retrieval-augmented generation (RAG) approaches remain limited in biomedical settings (see Fig. 1 a): (i) Few-shot prompting retrieves a few in-distribution exemplars (few-shot samples) and prompt the LLM to imitate them 17 ; 16 . While this can improve accuracy on similar cases, it does not yield structured explanations 22 . The link between examples is built via clustering and similarity search, which requires sizable, well-labeled exemplar data. As a result, exemplar RAG is brittle and biased: it optimizes for local pattern matching rather than concept coverage or causal explanations. (ii) Private knowledge databases consist of bespoke stores such as entity graphs 1 or schema-aligned tables 4 which can support step-wise explanations, but require strong priors and heavy maintenance, with limited reusability across domains. In biomedicine, where PubMed and MeSH (Medical Subject Headings) terms continue to grow, keeping such stores complete and current is prohibitively expensive. A parallel line of work equips LLMs with web search capabilities (“deep research”), producing responses with citations. Yet, these systems typically lack the ability to constrain retrieval to authoritative biomedical sources such as PubMed, often yielding citations from less reliable or incomplete domains. Recent “self-reflection” agents (Figure 1 b) provide another direction, using consistency checks 24 or reward-based signals 14 ; 21 to refine final answers. However, reflection occurs only after retrieval and response generation are complete, making the process computationally costly and unable to correct poor upstream retrieval or query formulation. Our approach. We present PubMed Reasoner, a multi-stage agent that mirrors the workflow of a biomedical researcher. Unlike prior methods that reflect only after producing responses, PubMed Reasoner introduces feedback much earlier. During query planning, a self-critic evaluates candidate MeSH terms and their Boolean composition directly against live PubMed metadata, eliminating the need for static private databases. This structured feedback prevents low-quality queries from propagating downstream and balances recall with precision. The refined query is then issued to PubMed, where a reflective retriever processes articles in small batches and halts once sufficient evidence is gathered, controlling token usage. Finally, PubMed Reasoner synthesizes an evidence-grounded response with explicit inline citations, ensuring transparency and interpretability. Instead of treating the LLM as a one-shot generator, PubMed Reasoner introduces a dynamic reasoning-based paradigm , orchestrating three interconnected stages over external evidence. This work makes the following contributions: • We shift biomedical QA beyond one-shot generation by introducing an iterative workflow that plans, retrieves, and reasons over external evidence. • We integrate self-critic query refinement with batch-wise reflective retrieval over the PubMed database, enabling robust evidence grounding without maintaining private knowledge stores. • We demonstrate consistent improvements in accuracy, explanation quality, and computational efficiency on PubMedQA and MMLU Clinical Knowledge, outperforming strong LLM, RAG and self-reflection baselines. 2 Preliminaries PubMed and MeSH Terms. PubMed is the primary biomedical literature database, indexing over 35 million references. Each article is annotated with Medical Subject Headings (MeSH), a controlled vocabulary that organizes biomedical concepts into a hierarchical taxonomy. Queries often combine MeSH terms with Boolean operators (AND, OR, NOT), enabling structured retrieval. Biomedical Question Answering. The input is a natural language question Q (e.g., “ Do leukotrienes play a key role in asthma? ”), and the output is a response R that is both factually accurate and explicitly supported by authoritative literature. Problem Setup. Given a user question Q , an optional task specification T (e.g., “ answer yes/no with a justification ”), and optional context C , the objective is to retrieve a set of relevant biomedical articles A and synthesize a final response R that is evidence-grounded and interpretable. Search Result Assessment Metrics. To enable iterative query refinement, we adapt evaluation dimensions from search benchmarks 9 ; 11 to the biomedical QA setting, defining three structured feedback signals: • Coverage: Does the MeSH term retrieve articles that represent the core biomedical concept? • Alignment: Are the retrieved articles relevant to the original question? • Redundancy: Does the MeSH term overlap with others, reducing retrieval efficiency? These signals guide improvements to the evolving query, ensuring that downstream reasoning is grounded in a high-quality evidence pool. 3 Proposed Method Figure 2: PubMed Reasoner stages. (1) Search with Self-Critic Query Refinement. From a user question, MeSH terms and a structured query are proposed. Self-critic evaluates each term for coverage , alignment , and redundancy , iteratively refining the query. (2) Reflective Article Retrieval with Early Stopping. PubMed Reasoner queries PubMed, filters results by title/abstract, and extracts supporting evidence in batches, checking whether accumulated evidence sufficiently covers the question; if so, retrieval is terminated early to save tokens and avoid unnecessary processing. (3) Evidence-Grounded Response Generation. Retained evidence is synthesized into final answer with explicit inline citations, ensuring factual grounding and traceability. More case studies are provided in App. G . We introduce PubMed Reasoner, a multi-stage agent framework inspired by the workflow of a biomedical researcher. Unlike direct LLM-based answering or one-shot retrieval-augmented generation, PubMed Reasoner explicitly separates reasoning into three phases: (1) Search with Self-Critic Query Refinement , (2) Reflective Article Retrieval with Early Stopping , and (3) Evidence-Grounded Response Generation . This design ensures that the system does not rely solely on parametric memory, but grounds its explanation in authoritative biomedical literature. 3.1 Search with Self-Critic Query Refinement Query Generation. The first stage constructs a structured