Efficient Document Parsing via Parallel Token Prediction
Lei Li, Ze Zhao, Meng LiZhongwang Lun, Yi Yuan, Xingjing Lu, Zheng Wei, Jiang Bian, Zang Li · 2026
Read Efficient Document Parsing via Parallel Token Prediction by Lei Li, Ze Zhao, Meng LiZhongwang Lun, Yi Yuan, Xingjing Lu, Zheng Wei, Jiang Bian, Zang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15206.txt.
Introduction
Li and co-authors speed up document parsing by teaching vision-language models to predict several tokens in parallel. Normal autoregressive decoding emits one token at a time, which bottlenecks parsing speed. Their method, Parallel-Token Prediction, inserts learnable tokens into the input and trains the model to use them for lookahead. A data pipeline supplies large-scale, high-quality parsing examples. On OmniDocBench and olmOCR-bench, decoding ran up to about twice as fast. Hallucinations dropped as well, and the method generalized across models.
Catalog record
| CREATOR | Lei Li, Ze Zhao, Meng LiZhongwang Lun, Yi Yuan, Xingjing Lu, Zheng Wei, Jiang Bian, Zang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42686 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15206
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Document Parsing via Parallel Token Prediction Lei Li, Ze Zhao, Meng LiZhongwang Lun, Yi Yuan, Xingjing Lu, Zheng Wei, Jiang Bian, Zang Li Abstract Document parsing, as a fundamental yet crucial vision task, is being revolutionized by vision-language models (VLMs). However, the autoregressive (AR) decoding inherent to VLMs creates a significant bottleneck, severely limiting parsing speed. In this paper, we propose Parallel-Token Prediction (PTP) , a plugable, model-agnostic and simple-yet-effective method that enables VLMs to generate multiple future tokens in parallel with improved sample efficiency. Specifically, we insert some learnable tokens into the input sequence and design corresponding training objectives to equip the model with parallel decoding capabilities for document parsing. Furthermore, to support effective training, we develop a comprehensive data generation pipeline that efficiently produces large-scale, high-quality document parsing training data for VLMs. Extensive experiments on OmniDocBench and olmOCR-bench demonstrate that our method not only significantly improves decoding speed (1.6 × \times -2.2 × \times ) but also reduces model hallucinations and exhibits strong generalization abilities. † † footnotetext: † Corresponding author. † † footnotetext: Code is available at https://github.com/flow3rdown/PTP-OCR . 1 Introduction Document parsing, also known as document content extraction 49 , aims to transform unstructured or semi-structured documents into structured, machine-readable outputs. This process involves accurately identifying and reconstructing diverse elements including text, images, formulas, and tables while preserving their logical ordering and hierarchical relationships as presented in the original documents. As a cornerstone task in multimodal understanding, document parsing plays a critical role in enabling advanced applications such as Retrieval-Augmented Generation (RAG) 20 ; 48 , document analysis 37 ; 2 , and data management 3 ; 44 , establishing a solid foundation for enabling machines to comprehend the digital world. Early document parsing methods predominantly adopted pipeline-based approaches 8 ; 41 ; 28 ; 30 , which decomposed the task into sequential modules, suffering from error accumulation and limited end-to-end optimization. With recent advances in Vision-Language Models (VLMs), an increasing number of methods have begun leveraging VLMs to revolutionize the document parsing task, either through end-to-end generation 4 ; 16 ; 44 ; 30 or by integrating VLMs into specific pipeline stages 6 ; 12 ; 26 ; 23 for improved multi-element recognition. However, as a real-world application-oriented task, document parsing demands not only high accuracy but also efficient processing speed, particularly for large-scale deployment scenarios. While VLMs have achieved remarkable improvements in parsing quality, their inherent autoregressive (AR) generation mechanism with next-token prediction (NTP) introduces a singnificant efficiency bottleneck. Recent efforts have explored various optimization strategies to accelerate VLM-based parsing, including output sequence compression 26 , visual token reduction 44 , and model parameter pruning 36 . Despite these advances, the sequential generation paradigm remains the inherent bottleneck , as the autoregressive decoding process leading to substantial latency that grows proportionally with document complexity and content density. Considering that the essence of OCR tasks lies in accurate transcription rather than semantic understanding, we can naturally decompose an image into multiple patches and perform parallel content recognition. This raises a natural question: Can this parallel recognition capability be inherently embedded within the model itself? To address this challenge, we propose Parallel Token Prediction (PTP) , a novel training and inference framework that breaks the sequential generation bottleneck by enabling models to produce multiple tokens per decoding step. Specifically, we insert some learnable register tokens 9 ; 13 into training sequences and optimize them to predict future tokens based on their positions. During inference, by appending N special tokens to the input sequence, the model generates N tokens in parallel within each decoding step, achieving theoretical N -fold acceleration. Extensive experiments on the OmniDocBench 27 dataset validate that PTP delivers significant throughput improvements while preserving model accuracy: PTP-1 attains 1.6 \times throughput over the NTP baseline, and PTP-2 achieves 2.2 \times acceleration. Furthermore, we generalize PTP to broader vision-language understanding tasks and synergize it with speculative decoding 18 ; 34 , resulting in an impressive 82% acceptance ratio. To summarize, our key contributions are encapsulated as follows: (1) We propose Parallel Token Prediction (PTP), a model-agnostic, plugable, and highly efficient acceleration method for document parsing. PTP achieves 1.6 \times -2.2 \times throughput improvements without compromising accuracy. (2) We construct a high-quality layout-level document parsing dataset through an automated generation framework that integrates multiple types of VLMs for data annotation, coupled with sophisticated filtering and deduplication strategies to ensure data quality. (3) We conduct comprehensive analyses and ablation studies, validating the effectiveness of PTP and further exploring its potential in vision-language understanding (VLU) scenarios. 2 Related Work Document Parsing Approaches. Document parsing methods can be broadly categorized into two approaches: (i) Pipeline-based Approaches: These methods 28 ; 41 decompose document parsing into sequential modular tasks, including layout analysis, text, formula and table recognition, and reading order detection, etc. Each module employs a specialized model optimized for its specific task. While enabling fine-grained optimization and interpretability, they suffer from error accumulation across stages and exhibits degraded performance in challenging or domain-specific scenarios. (ii) VLM-based Approaches: These methods leverage general or domain-specific vision-language models to replace multiple modular components, thereby simplifying the parsing pipeline. Early works 4 ; 42 ; 43 introduce end-to-end OCR-free VLMs that directly parse document images, eliminating error propagation, but remain constrained by scalability and efficiency concerns. Recent approaches 12 ; 19 ; 26 adopt a hybrid strategies combining layout analysis with VLM-based recognition. While this strategy effectively leverages both the efficiency of pipeline methods and the accuracy of VLMs, two critical limitations persist: (1) autoregressive decoding inherently limits parsing speed, and (2) the scarcity of large-scale, high-quality training data poses challenges for model development. Efficient Document Parsing. While autoregressive models improve OCR accuracy and robustness, efficiency remains a critical bottleneck. Existing acceleration approaches can be categorized as follows: (i) Multi-Token Prediction: Early works 11 ; 33 ; 10 employ non-autoregressive (NAR) vision-language models trained with Connectionist Temporal Classification (CTC) loss to achieve multi-token prediction. However, these methods require complex architectural modifications, exhibit limited performance, and are restricted to span-level OCR tasks, failing to scale to paragraph- or document-level parsing. Recent efforts 14 ; 21 introduce auxiliary MTP heads to enable multi-token prediction in language models, but their applications to document parsing remain unexplored. (ii) Sequence Compression: Recent studies reduce computational cost by shortening input or output sequences. 26 designs compact representation languages for formulas and tables to reduce output tokens, thereby improving throughput. 44 proposes DeepEncoder to compress visual representations, reducing input tokens and accelerating prefill stage. 36 prunes redundant vocabulary tokens to decrease model capacity and decoding overhead. While achieving moderate efficiency gains, these methods do not fundamentally address the autoregressive (AR) decoding bottleneck. In this work, we propose Parallel Token Prediction (PTP), which enables parallel decoding in VLMs without sacrificing performance. PTP is model-agnostic and orthogonal to existing architectures and acceleration techniques, delivering substantial improvements in parsing efficiency. Figure 1 : An illustration of the data creation pipeline. 3 Dataset Engine Current document parsing and OCR datasets mainly focus on span-level or file-level annotations, with a critical shortage of layout-level data. Moreover, existing datasets exhibit limited diversity in document types and difficulty levels, hindering model generalization to real-world scenarios. To address