When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering

Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar · 2026

Read When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering by Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21807.txt.

Description

Doeun Lee and co-authors introduce OGCaReBench, a retrieval benchmark for clinical questions that fall outside evidence-based guidelines. Guidelines cover studied diagnostic and treatment paths, but the long tail of real care escapes them, and medical models mostly memorize common guideline knowledge. Built from published case reports and validated by medical experts, the benchmark poses long-form, free-text clinical questions about rare cases. Even the best baseline answers only fifty-six percent correctly, while retrieved medical articles lift performance to eighty-two percent, showing evidence grounding matters.

Catalog record

CREATORDoeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82381
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21807

Browse the full library · About the Tower of Babel