When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering
Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar · 2026
Read When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering by Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21807.txt.
Description
Doeun Lee and co-authors introduce OGCaReBench, a retrieval benchmark for clinical questions that fall outside evidence-based guidelines. Guidelines cover studied diagnostic and treatment paths, but the long tail of real care escapes them, and medical models mostly memorize common guideline knowledge. Built from published case reports and validated by medical experts, the benchmark poses long-form, free-text clinical questions about rare cases. Even the best baseline answers only fifty-six percent correctly, while retrieved medical articles lift performance to eighty-two percent, showing evidence grounding matters.
Catalog record
| CREATOR | Doeun Lee, Muge Zhang, Yi Yu, Ashish Manne, Stephen Koesters, Frank Wen, Brady Buchanan,, Lynda Villagomez, Oluwatoba Moninuola, James Lim, Kathryn Tobin,, Andrew Srisuwananukorn, Ping Zhang, Sachin Kumar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82381 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.21807