How Far Is Document Parsing from Solved? Pure DocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings

Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected] · 2026

Read How Far Is Document Parsing from Solved? Pure DocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings by Zhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07492.txt.

Introduction

Zhiheng Li and fourteen co-authors built a benchmark to measure how well computers can parse documents into text, formulas, tables, and reading order. Existing benchmarks, they argue, are too narrow to show what current parsers can really do. Their benchmark, PureDocBench, covers ten domains and sixty-six subcategories across nearly fifteen hundred pages.

Each page appears in clean, digitally degraded, and real photographed or shared versions, with annotations checked by automated validation and human review. The team tested fifty-eight models, from pipeline parsers to general vision-language models. The best average score was about eighty out of one hundred, and real-world versions cost models roughly twelve points against clean ones. All data, tools, and evaluation code are public.

Catalog record

CREATORZhiheng Li, Zongyang Ma, Jiaxian Chen, Jianing Zhang, Zhaolong Su, Yutong Zhang Zhiyin Yu Ruiqi Liu Xiaolei Lv Bo Li, Jun Gao Ziqi Zhang Chunfeng Yuan Bing Li Weiming Hu, [6pt] CASIA, [4pt] Core contributors. Corresponding author., [2pt] [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE157 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07492

Browse the full library · About the Tower of Babel