irpapers : A Visual Document Benchmark for Scientific Retrieval and Question Answering

Connor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt · 2026

Read irpapers : A Visual Document Benchmark for Scientific Retrieval and Question Answering by Connor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17687.txt.

Description

Connor Shorten and team introduce IRPAPERS, a visual document benchmark for scientific retrieval and question answering. Text and relational processing have surged, yet visual document processing lags. Multimodal models now read document images directly, raising a key question: how do image-based systems stack up against text-based ones?

The benchmark holds over three thousand pages from well over a hundred scientific papers, each page paired with an OCR transcription, plus one hundred eighty needle-in-the-haystack questions. Text retrieval and image retrieval land close on recall, with complementary failures, so a multimodal hybrid tops either alone. For question answering, text-based retrieval-augmented systems align better with ground truth than image-based ones, and both gain much from deeper retrieval.

Catalog record

CREATORConnor Shorten, Augustas Skaburskas, Daniel M. Jones, Charles Pierse, Roberto Esposito, John Trengrove, Etienne Dilocker, Bob van Luijt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17687

Browse the full library · About the Tower of Babel