irpapers : A Visual Document Benchmark for Scientific Retrieval and Question Answering
Connor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt · 2026
Read irpapers : A Visual Document Benchmark for Scientific Retrieval and Question Answering by Connor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17687.txt.
Description
Connor Shorten and team introduce IRPAPERS, a visual document benchmark for scientific retrieval and question answering. Text and relational processing have surged, yet visual document processing lags. Multimodal models now read document images directly, raising a key question: how do image-based systems stack up against text-based ones?
The benchmark holds over three thousand pages from well over a hundred scientific papers, each page paired with an OCR transcription, plus one hundred eighty needle-in-the-haystack questions. Text retrieval and image retrieval land close on recall, with complementary failures, so a multimodal hybrid tops either alone. For question answering, text-based retrieval-augmented systems align better with ground truth than image-based ones, and both gain much from deeper retrieval.
Catalog record
| CREATOR | Connor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-17687