Doc- V ∗ V^{*} : Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA

Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai · 2026

Read Doc- V ∗ V^{*} : Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA by Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13731.txt.

Introduction

Zheng and eleven colleagues present Doc-V*, an OCR-free agentic tool for questions over multi-page documents. Long, dense documents strain end-to-end models. Passive retrieval pipelines break easily. Doc-V* starts from a thumbnail overview. It then navigates by semantic retrieval and page fetching. Evidence gathers in a working memory that grounds its reasoning. Trained by imitation from expert paths and refined with group relative policy optimization, it beats open-source baselines on five benchmarks. Out-of-domain performance rises by nearly 48 percent over a retrieval baseline.

Catalog record

CREATORYuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73336
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13731

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Doc- V ∗ V^{*} : Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai Abstract Multi-page Document Visual Question Answering requires reasoning over semantics, layouts, and visual elements in long, visually dense documents. Existing OCR-free methods face a trade-off between capacity and precision: end-to-end models scale poorly with document length, while visual retrieval-based pipelines are brittle and passive. We propose Doc- V ∗ V^{*} , an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation. Doc- V ∗ V^{*} begins with a thumbnail overview, then actively navigates via semantic retrieval and targeted page fetching, and aggregates evidence in a structured working memory for grounded reasoning. Trained by imitation learning from expert trajectories and further optimized with Group Relative Policy Optimization, Doc- V ∗ V^{*} balances answer accuracy with evidence-seeking efficiency. Across five benchmarks, Doc- V ∗ V^{*} outperforms open-source baselines and approaches proprietary models, improving out-of-domain performance by up to 47.9% over RAG baseline. Other results reveal effective evidence aggregation with selective attention, not increased input pages. 1 Introduction Understanding multi-page, visually rich documents—such as academic papers, financial reports, and industrial manuals—remains a core challenge in Document Visual Question Answering (DocVQA) Mathew et al. 2021 ; Tito et al. 2023 . Unlike plain text, such documents convey information through a complex interplay of textual semantics, spatial layouts, and visual elements (e.g., tables and figures) Ding et al. 2025 . Conventional OCR-based pipelines linearize document images into text before reasoning Memon et al. 2020 ; Wang et al. 2024 ; Appalaraju et al. 2021 , but inevitably lose fine-grained layout cues and suffer from cascading OCR errors. Recent OCR-free or pure-vision approaches instead model documents directly as images using multimodal large language models (MLLMs) Lee et al. 2023 ; Kim et al. 2022 ; Liu et al. 2024b , enabling joint visual–semantic reasoning and improved robustness. Figure 1: The Doc- V^{*} agent workflow for multi-page document VQA. It adopts an active perception paradigm by planning from a global thumbnail view and iteratively deciding when to fetch high-resolution pages or perform semantic searches, aggregating evidence in a structured working memory for grounded answering. However, existing pure-vision methods face a fundamental trade-off between capacity and precision . End-to-end models process entire documents as long image sequences Zhu et al. 2025 ; Hu et al. 2025 ; Bai et al. 2025 , but scale poorly to long documents due to quadratic attention cost, context length limits, and the " lost-in-the-middle " effect Liu et al. 2024a . In contrast, visual retrieval-augmented generation ( RAG ) systems reduce noise by retrieving top- k pages before generation Cho et al. 2024 ; Faysse et al. 2025 ; Song et al. 2025 , yet suffer from retrieval errors, sensitivity to hyperparameters, and limited multi-hop reasoning. Critically, both paradigms remain passive : they process a fixed input without adapting their strategy as new evidence emerges. We argue that this limitation arises from a mismatch with human document-reading behavior. Guided by Active Vision Theory Aloimonos et al. 1988 , human experts treat perception as a goal-directed process: they first obtain a global structural overview, then iteratively seek, verify, and integrate evidence while maintaining working memory. Inspired by this cognitive process, we propose Doc- V^{*} , an OCR-free agentic framework that formulates multi-page DocVQA as a sequential evidence aggregation process . Doc- V^{*} begins with a Global Thumbnail Overview that provides a low-cost structural prior, and then alternates between structured visual reasoning and document navigation actions , including semantic retrieval and targeted page fetching. This interactive reasoning allows the agent to active perception and piece together discontinuous visual evidence before answering. Figure 1 shows the agent workflow of Doc- V^{*} . To train Doc- V^{*} , we adopt a two-stage optimization strategy. We first perform supervised fine-tuning using high-quality interaction trajectories synthesized by GPT-4o, providing a strong cold start. We then apply Group Relative Policy Optimization (GRPO) Guo et al. 2025 to jointly optimize answer accuracy and evidence-seeking efficiency through reward signals that account for answer quality, evidence discovery, and format compliance. Extensive experiments on five benchmarks demonstrate that Doc- V^{*} consistently outperforms existing open-source baselines and rivals proprietary models like GPT-4o , particularly in out-of-domain settings where it achieves up to a 47.9% improvement over static RAG baselines, as well as robustness under variations in retrieval tools and hyperparameters. We also demonstrate that long-document understanding hinges on effective aggregation of evidence with selective attention rather than sheer input pages , which is crucial to the success of Doc- V^{*} . 2 Related Work Visual Document Question Answering (DocVQA) has progressed from single-page inputs to long and multi-page documents, driven by the increasing demand for handling complex real-world document understanding scenarios. Existing methods mainly follow two paradigms: 1) OCR-based DocVQA OCR-based approaches first extract textual and layout structures via OCR and document parsing, followed by reasoning over structured representations Tito et al. 2023 ; Zhang et al. 2024 ; Luo et al. 2024 ; Fujitake 2024 ; Li et al. 2024 ; Duan et al. 2025 ; Nacson et al. 2025 ; Xie et al. 2024 . While effective on clean and well-formatted documents, these pipelines inevitably suffer from cascading OCR and layout errors and generalize poorly to noisy or out-of-domain scenarios; 2) OCR-free Pure-Vision DocVQA Recent OCR-free methods leverage large vision–language models to reason directly over document images, preserving rich visual and spatial cues. However, scaling to long documents remains challenging. Existing approaches include: (i) end-to-end models that process all pages jointly Hu et al. 2025 ; Zhu et al. 2025 , which scale poorly with document length, as computational cost and memory consumption grow rapidly with the number of pages; (ii) retrieval-based methods that select top- k pages before generation Cho et al. 2024 ; Yu et al. 2024 ; Chen et al. 2024 ; Tanaka et al. 2025 ; Wang et al. 2025 ; Wu et al. 2025 ; Shi et al. 2025 , improving efficiency but remaining sensitive to retrieval errors and fixed hyperparameters; and (iii) agent-based systems that iteratively explore documents Xu et al. 2025 ; Yang et al. 2025 ; Yue et al. 2025 , which introduce interaction at the cost of increased complexity. In contrast, our method formulates DocVQA as a sequential evidence aggregation process, enabling a single OCR-free agent to actively and efficiently aggregate visual evidence over long documents. Figure 2: Overview of the training pipeline for Doc- V^{*} . (a) Training data construction. Documents and queries are paired to generate thumbnail-guided reasoning trajectories, followed by quality filtering. (b) Supervised fine-tuning (SFT). (c) Reinforcement learning with GRPO. 3 Method 3.1 Formulation and Cognitive Motivation Faced with lengthy, unfamiliar documents, human experts exhibit pronounced goal-directedness and proactivity rather than reading cover-to-cover: they navigate using structural cues and keyword-like searches, and iteratively update their strategy as evidence is found. This behavior is consistent with Active Vision Aloimonos et al. 1988 , which views perception as goal-directed sampling to reduce uncertainty, and Resource-Rational Cognition Lieder and Griffiths 2020 , which trades off information gain against processing costs. Motivated by these

Read the complete text