How Far Is Document Parsing from Solved? Pure DocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected] · 2026
Read How Far Is Document Parsing from Solved? Pure DocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings by Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07492.txt.
Introduction
Zhiheng Li and fourteen co-authors built a benchmark to measure how well computers can parse documents into text, formulas, tables, and reading order. Existing benchmarks, they argue, are too narrow to show what current parsers can really do. Their benchmark, PureDocBench, covers ten domains and sixty-six subcategories across nearly fifteen hundred pages.
Each page appears in clean, digitally degraded, and real photographed or shared versions, with annotations checked by automated validation and human review. The team tested fifty-eight models, from pipeline parsers to general vision-language models. The best average score was about eighty out of one hundred, and real-world versions cost models roughly twelve points against clean ones. All data, tools, and evaluation code are public.
Catalog record
| CREATOR | Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 157 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07492