G 2 G^{2} -Reader: Dual Evolving Graphs for Multimodal Document QA

Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen · 2026

Read G 2 G^{2} -Reader: Dual Evolving Graphs for Multimodal Document QA by Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22055.txt.

Introduction

Yaxin Du and co-authors build a question answering system for long, mixed documents. Retrieval-augmented generation answers questions over long documents, but it stays brittle when text, tables, and figures are interleaved across many pages. Flat chunking, the usual preparation, breaks the document's native structure.

Chunks cut loose from their tables and figures become fragments that are hard to interpret alone. The team's 2026 preprint proposes a reader built on dual evolving graphs, structures that track both the document's layout and its meaning as reading proceeds. Reports, manuals, and papers are inherently multimodal, so a reader that respects that structure could answer with far fewer confident mistakes.

Catalog record

CREATORYaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22055

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

G 2 G^{2} -Reader: Dual Evolving Graphs for Multimodal Document QA Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen Abstract Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal alignment, yielding semantic fragments that are hard to interpret in isolation. Second, even iterative retrieval can fail in long contexts by looping on partial evidence or drifting into irrelevant sections as noise accumulates, since each step is guided only by the current snippet without a persistent global search state. We introduce G 2 G^{2} -Reader, a dual-graph system, to address both issues. It evolves a Content Graph to preserve document-native structure and cross-modal semantics, and maintains a Planning Graph, an agentic directed acyclic graph of sub-questions, to track intermediate findings and guide stepwise navigation for evidence completion. On VisDoMBench across five multimodal domains, G 2 G^{2} -Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%). † † affiliationnotice: Equal contribution https://github.com/DorothyDUUU/G2_Reader Figure 1 : Overview of the G^{2} -Reader framework (Left). G^{2} -Reader bridges the gap between offline document indexing and online inference through a dual-graph architecture: (Right) The Content Graph ( \mathcal{G}_{C} ) provides a structured multimodal representation of documents with evolved node attributes; (Middle) The Planning Graph ( \mathcal{G}_{P} ) serves as an explicit, dynamic reasoning state that iteratively orchestrates sub-question decomposition and evidence assembly. 1 Introduction Large language models (LLMs) ( 48 ) have demonstrated remarkable capabilities in assisting humans with reading, analysis, and reasoning over complex content. However, real-world reading tasks frequently involve long documents or large corpora that exceed the limited context window of LLMs ( 2 ; 42 ) . To address this constraint, Retrieval-Augmented Generation (RAG) ( 21 ) has emerged as a dominant paradigm, transforming long-context understanding into a retrieve-then-read workflow by grounding generation on externally retrieved evidence ( 17 ; 13 ; 20 ) . In practice, documents are often multimodal, interleaving natural language with tables and figures over many pages. A key feature of such documents is that they encode tightly coupled heterogeneous evidence ( e.g. , text, tables, and figures) that mutually ground each other ( 34 ; 6 ) . This inherent complexity poses a fundamental crisis for current RAG systems, manifesting in two coupled dimensions: ❶ the Representation Challenge: Despite this coupling, standard RAG pipelines flatten documents into independent chunks, which breaks document-native structure and cross-modal alignment ( e.g. , separating a figure/table from its caption, referring text, and explanatory paragraph) ( 9 ) . As a result, retrieved contexts are often semantic fragments that are hard to interpret in isolation. ❷ the Retrieval Challenge: Traditional RAG is largely one-shot, which is brittle for long contexts; iterative retrieval ( 18 ; 36 ) was introduced to refine queries step by step. In long contexts, iterative retrieval often fails by looping on partial evidence or drifting into irrelevant sections as noise accumulates ( 25 ; 10 ) . This happens because each step is guided only by the current snippet, with no persistent global state of the search. To tackle these two problems, we introduce G^{2} -Reader , a dual- G raph system for long-document multimodal QA. Its key idea is that RAG should advance on both evidence representation and retrieval-reasoning fronts, and we achieve this through a dual-graph mechanism: ❶ Content Graph ( \mathcal{G}_{C} ): Instead of flat chunking, we ground multimodal evidence in a structured topology that preserves document-native layouts and cross-modal alignments. It enables message passing, allowing evidence representations to evolve contextually and restore global awareness. ❷ Planning Graph ( \mathcal{G}_{P} ): We maintain an agentic directed acyclic graph (DAG) as an explicit structured carrier for the reasoning process. By decomposing complex questions into sub-questions, it iteratively updates the reasoning state with intermediate findings to guide precise navigation over the underlying content. Concretely, G^{2} -Reader first converts each document into a heterogeneous multimodal content graph whose nodes are (a) paragraphs and (b) multimodal elements (tables/figures with captions), and whose edges encode document-native relations such as reference and multimodal associations. Through lightweight message passing, each node perceives its broader context, establishing meaningful connections and preventing the semantic fragmentation common in traditional chunking. Secondly, over this stable structural foundation, G^{2} -Reader orchestrates reasoning via the Planning Graph, a DAG of sub-questions. Rather than a blind one-shot lookup, the system performs context-aware exploration: each retrieval step informs and refines the subsequent plan. This iterative feedback loop allows the system to incrementally assemble a coherent body of evidence, moving from initial uncertainty to a complete and logically sound answer. By grounding the dynamic reasoning path within a fixed structural prior, G^{2} -Reader ensures that the retrieved evidence is both structurally intact and logically sufficient. Empirical evaluations across five multimodal domains (including slides, web pages, academic papers, and textbooks) demonstrate G^{2} -Reader’s superiority and robustness. Utilizing the open-source Qwen3-VL-32B-Instruct, our system achieves an average score of 66.21%, significantly surpassing all baselines and even a standalone GPT-5 (53.08%). These results validate that our dual-graph architecture effectively empowers open-source models to outperform powerful closed-source frontiers by providing a structured foundation for complex evidence assembly. Our main contributions are: ❶We identify two coupled limitations of current multimodal RAG systems: fragmented evidence representation and unstable long-context retrieval, and therefore propose G^{2} -Reader, a dual-graph system to address both. ❷We introduce a Content Graph to preserve semantic structures within multimodal documents, and a Planning Graph to maintain reasoning state and guide stepwise navigation for evidence assembly. ❸On VisDoMBench across five domains, G^{2} -Reader with an open-source backbone achieves the best overall accuracy. Comprehensive ablations further validate the effectiveness and complementarity of each component. 2 Related Work 2.1 Memory Construction for RAG Retrieval-augmented generation (RAG) critically depends on how external memory is represented. Early approaches largely treat documents as flattened collections of text chunks, discarding document structure and multimodal context ( 21 ) . To overcome this limitation, a substantial line of work focuses on structuring document memory prior to retrieval, organizing content into graphs over entities, passages, or topics in order to encode relations such as co-reference, topical continuity, and cross-section dependency ( 8 ; 23 ; 4 ; 14 ; 33 ; 24 ; 43 ; 5 ) . In parallel, multimodal RAG systems extend structured memory to visually rich documents by retrieving or indexing over images, layouts, tables, and figures, or by aligning heterogeneous elements within multimodal graphs or unified indices ( 45 ; 9 ; 38 ; 11 ) . Despite the progress, memory construction in existing systems is typically treated as a single-pass preprocessing step, relying on heuristic extraction or standalone summarization, and retrieval predominantly matches queries against embeddings of raw textual or visual content. In contrast, our method formulates memory construction as an iterative, VLM-driven consolidation process , where multimodal memory units are evolved based on their neighborhood to distill core semantics and uncover document-intrinsic relations, and tightly coupled into a memory graph for more precise and comprehensive retrieval beyond surface-level similarity. 2.2

Read the complete text