pdfQA: Diverse, Challenging, and Realistic Question Answering over PDFs
Tobias Schimanski, Imene Kolli, Yu Fan, Ario Saeid Vaghefi, Jingwei Ni, Elliott Ash, Markus Leippold · 2026
Read pdfQA: Diverse, Challenging, and Realistic Question Answering over PDFs by Tobias Schimanski, Imene Kolli, Yu Fan, Ario Saeid Vaghefi, Jingwei Ni, Elliott Ash, Markus Leippold (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02285.txt.
Description
Tobias Schimanski and six co authors introduce pdfQA, a data set for testing whether computer systems can answer questions about real PDF files. PDFs are the second most used document type on the web, yet most question answering sets start from plain text or cover one narrow field. A PDF adds layout, tables, and mixed sources that plain text sets miss.
The set holds about two thousand human written question and answer pairs and a similar synthetic set, spread across many domains. Each pair is tagged along ten dimensions of difficulty, such as file type, where the answer sits, and what kind of answer is needed. Tests with open language models show clear failures that line up with those difficulty tags, making the set useful for checking full reading pipelines.
Catalog record
| CREATOR | Tobias Schimanski, Imene Kolli, Yu Fan, Ario Saeid Vaghefi, Jingwei Ni, Elliott Ash, Markus Leippold |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-02285