Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs
2026
Read Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10877.txt.
Introduction
Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy, and Siddhant Rai, writing as team Neural, present a single prompt-optimization method for clinical question answering over health records. The ArchEHR-QA 2026 task splits the job into four parts: interpreting questions, finding evidence, generating answers, and aligning answers to evidence.
Their Neural1.5 system breaks the pipeline into modular stages and tunes prompts for each stage automatically with DSPy's MIPROv2 optimizer. Self-consistency voting across repeated runs plus stage-specific checks, like self-reflection and chain-of-verification, further clean the output. Among teams entering all four subtasks, Neural placed second overall. Clinical QA builders get a cheaper rival to fine-tuning that still grounds answers in notes.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30577 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10877
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs Abstract Automated question answering (QA) over electronic health records (EHRs) demands precise evidence retrieval, faithful answer generation, and explicit grounding of answers in clinical notes. In this work, we present Neural1.5 , our method for the ArchEHR-QA 2026 shared task at CL4Health@LREC 2026, which comprises of four subtasks: question interpretation, evidence identification, answer generation, and evidence alignment. Our approach decouples the task into independent, modular stages and employs DSPy’s MIPROv2 optimizer to automatically discover high-performing prompts, jointly tuning instructions and few-shot demonstrations for each stage. Within every stage, self-consistency voting over multiple stochastic inference runs suppresses spurious errors and improves reliability, while stage-specific verification mechanisms (e.g., self-reflection and chain-of-verification for alignment) further refine output quality. Among all teams that participated in all four subtasks, our method ranks second overall (mean rank 4.00), placing 4th, 1st, 4th, and 7th on Subtasks 1–4, respectively. These results demonstrate that systematic, per-stage prompt optimization combined with self-consistency mechanisms is a cost-effective alternative to model fine-tuning for multi-faceted clinical QA. Keywords: clinical question answering, prompt optimization, electronic health records, large language models, evidence grounding, evidence alignment Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs Abrar Majeedi 1,∗ , Viswanatha Reddy Gajjala 1,∗ , Sai Prasanna Teja Reddy Bogireddy 2,∗ , Siddhant Rai 3,∗ 1 University of Wisconsin–Madison 2 University of Chicago 3 Independent Researcher {amajeedi, vgajjala}@wisc.edu , [email protected] ∗ Equal contribution Abstract content 1. Introduction Patient medical advice requests have surged 55% since 2019, with physicians now spending 24% more time on inbox management 1 . Automatically answering these questions using electronic health records (EHRs) could substantially reduce clinician burden, yet requires systems that not only generate accurate responses but also explicitly ground every claim in verifiable clinical evidence. The ArchEHR-QA 2026 shared task at CL4Health@LREC 2026 soni-etal-2026-archehr-qa takes a step towards addressing this challenge through four complementary subtasks: (1) transforming verbose patient questions into concise clinician-interpreted queries, (2) identifying minimal evidence sentences from clinical notes, (3) generating grounded answers, and (4) aligning each answer sentence to its supporting evidence. Together, these subtasks capture the complete pipeline from question understanding to explainable, evidence-backed clinical QA. The natural language processing capabilities of Large Language Models (LLMs) present a promising approach. Although LLMs have demonstrated strong performance in clinical QA 9 , their deployment faces two critical barriers. First, fine-tuning on clinical data is constrained by limited supervised datasets and overfitting risks. Second, prompt engineering for multi-stage clinical workflows remains labor-intensive and domain-specific, requiring expert iteration to identify optimal instructions and demonstrations 3 . Existing automated prompt optimization techniques 13 typically treat tasks holistically, failing to leverage the modular structure inherent in clinical QA pipelines where evidence identification, answer generation, and citation alignment each demand distinct reasoning patterns. In this work, we present Neural1.5 , a modular LLM method that addresses all four ArchEHR-QA 2026 subtasks through automated, stage-specific prompt optimization . Our key design principle is systematic task decomposition: by defining clear evaluation objectives for each subtask, we enable DSPy’s MIPROv2 optimizer 4 to automatically discover high-performing prompts that jointly tune instructions and few-shot demonstrations. We further enhance reliability through self-consistency voting 14 for evidence identification and a three-stage alignment pipeline combining initial alignment, self-reflection to prune false positives, and chain-of-verification with confidence-weighted majority voting. Our proposed method achieves competitive results across all four subtasks, ranking 4th (Subtask 1), 1st (Subtask 2 with Strict Micro F1 of 63.7), 4th (Subtask 3), and 7th (Subtask 4), with a mean rank of 4.00 across all subtasks— second overall among teams participating in all four subtasks. These results demonstrate that systematic, per-stage prompt optimization combined with self-consistency mechanisms offers a cost-effective alternative to model fine-tuning for multi-faceted clinical QA. The key contributions of our work are: • Full-Pipeline Automated Optimization: We propose the method to apply automated prompt optimization independently to all four ArchEHR-QA subtasks, with stage-specific objectives that capture distinct reasoning requirements for question interpretation, evidence retrieval, answer generation, and citation alignment. • Three-Stage Alignment with Self-Reflection: We introduce an alignment pipeline that combines initial sentence-level alignment, self-reflection to identify and remove spurious citations, and chain-of-verification, with confidence-weighted majority voting across multiple stochastic runs to suppress hallucinations/errors. • Consistent Multi-Task Performance: With a mean rank of 4.00 across all four subtasks, we demonstrate that modular, optimization-driven approaches can maintain strong performance across diverse clinical QA challenges, offering a practical blueprint for deploying LLMs in safety-critical healthcare applications. 2. Related Work Clinical QA: Developing QA systems for clinical data has long been an interest in biomedical NLP. Earlier datasets like emrQA 7 generated large-scale QA pairs from electronic medical records. Recent research has shown that LLMs can achieve near-expert performance on medical QA benchmarks 9 . The ArchEHR-QA shared task 11 ; 12 advances this line of work by requiring systems to ground answers explicitly in clinical notes. Our prior work 2 demonstrated the effectiveness of automated prompt optimization for a single task, achieving second place overall. Building on these insights, the 2026 edition expands to four complementary subtasks spanning question interpretation, evidence identification, answer generation, and evidence alignment. Prompt Optimization: There is growing interest in automated prompt search. Methods such as APE 20 and OPRO 16 treat prompt design as a black-box optimization problem. MIPRO 6 extends this to multi-stage LLM programs, jointly optimizing instructions and demonstration examples. Our work leverages MIPROv2 6 , which uses a combination of prompt proposal and Bayesian search. Self-Consistency: LLMs can produce variable outputs given the same prompt. The self-consistency decoding strategy 14 addresses this by sampling multiple outputs and selecting the most consistent result. 3. Task Description The ArchEHR-QA 2026 shared task 11 comprises four subtasks. The dataset 10 consists of patient-authored questions, clinician-interpreted counterparts, clinical note excerpts with sentence-level relevance annotations, and reference clinician-authored answers with answer–evidence alignments. Subtask 1: Question Interpretation. Given a free-text patient question, generate a concise clinician-interpreted question ( \leq 15 words) that captures the core clinical information need. Subtask 2: Evidence Identification. Given the patient question, clinician question, and a clinical note excerpt with numbered sentences, identify the minimal set of note sentences that provide evidence needed to answer the question. Subtask 3: Answer Generation. Given the questions and clinical note excerpt, generate a grounded natural-language answer ( \leq 75 words) using only information from the notes. Subtask 4: Evidence Alignment. Given the questions, clinical note excerpt, and a reference answer with numbered sentences, align each answer sentence to its supporting note sentence(s). 4. Methodology Our method draws on a human-inspired decoupling strategy, separating question understanding, evidence gathering, answer formulation, and evidence attribution into distinct stages. We operationalize this intuition as a modular pipeline, with each subtask addressed by a DSPy program whose prompts are optimized independently. The initial prompt templates (DSPy signatures) for all subtasks are provided in Appendix A . 4.1. Subtask 1: Question Interpretation We define a DSPy Signature for question interpretation that instructs the LLM to transform a patient narrative into a concise clinician-interpreted question. The signature encodes domain-specific heuristics: preserving key medical terms exactly (procedure names, medication names, condition names), using patient-specific phrasing (“him/her/the patient”), and