SciEGQA: A Dataset for Scientific Evidence-Grounded Question Answering and Reasoning

Wenhan Yu, Zhaoxi Zhang, Wang Chen, Guanqiang Qi, Werikang Li, Lei Sha, Deguo Xia, Jizhou Huang · 2025

Read SciEGQA: A Dataset for Scientific Evidence-Grounded Question Answering and Reasoning by Wenhan Yu, Zhaoxi Zhang, Wang Chen, Guanqiang Qi, Werikang Li, Lei Sha, Deguo Xia, Jizhou Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15090.txt.

Introduction

Wenhan Yu, Zhaoxi Zhang, and six co-authors build SciEGQA, a data set that tests whether AI systems can answer questions about scientific papers using the right evidence. Scientific documents mix text, figures, and tables, so finding the exact region that supports an answer is hard. Most benchmarks grade only the final answer, leaving the evidence unmarked and the reasoning unchecked.

SciEGQA marks the supporting evidence as coherent document regions with boxes around them. One part is a human-annotated benchmark of high-quality question-answer pairs; the other is a much larger training set built automatically. Systems can now be scored on grounding their answers, not just guessing them, which matters for trustworthy research assistants.

Catalog record

CREATORWenhan Yu, Zhaoxi Zhang, Wang Chen, Guanqiang Qi, Werikang Li, Lei Sha, Deguo Xia, Jizhou Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-15090

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SciEGQA: A Dataset for Scientific Evidence-Grounded Question Answering and Reasoning Wenhan Yu, Zhaoxi Zhang, Wang Chen, Guanqiang Qi, Werikang Li, Lei Sha, Deguo Xia, Jizhou Huang Abstract . Scientific documents contain complex multimodal structures, which makes evidence localization and scientific reasoning in Document Visual Question Answering particularly challenging. However, most existing benchmarks evaluate models only at the page level without explicitly annotating the evidence regions that support the answer, which limits both interpretability and the reliability of evaluation. To address this limitation, we introduce SciEGQA , a scientific document question answering and reasoning dataset with semantic evidence grounding , where supporting evidence is represented as semantically coherent document regions annotated with bounding boxes. SciEGQA consists of two components: a human-annotated fine-grained benchmark containing 1,623 high-quality question–answer pairs, and a large-scale automatically constructed training set with over 30K QA pairs generated through an automated data construction pipeline. Extensive experiments on a wide range of Vision-Language Models (VLMs) show that existing models still struggle with evidence localization and evidence-based question answering in scientific documents. Training on the proposed dataset significantly improves the scientific reasoning capabilities of VLMs. The project page is available at https://yuwenhan07.github.io/SciEGQA-project/ . DOI: XXXXXXX.XXXXXXX CCS: Information systems Multimedia databases CCS: Computing methodologies Computer vision tasks CCS: Computing methodologies Visual content-based indexing and retrieval 1. Introduction With the rapid development of Multimodal Large Language Models (MLLMs) ( Li et al., 2024b ; Wu et al., 2024 ; Yin et al., 2024 ; Caffagni et al., 2024 ) , Vision–Language Models (VLMs) have achieved remarkable progress in tasks such as image understanding, visual question answering, and multimodal reasoning ( Wang et al., 2024a ) . However, compared with natural images, scientific documents present fundamentally different challenges. A page of a scientific paper often contains dense multimodal content, including textual paragraphs, tables, figures, and cross-page references ( Wang et al., 2024b ; Li et al., 2024a ) . These characteristics make Document Visual Question Answering (DocVQA) ( Mathew et al., 2021 ) particularly challenging, as models must not only accurately localize the relevant evidence regions but also integrate information across modalities to perform reasoning. In recent years, a variety of DocVQA benchmarks have been proposed to evaluate document understanding capabilities ( Mathew et al., 2021 ; Tito et al., 2023 ; Mathew et al., 2022 ; Saikh et al., 2022 ) . Nevertheless, existing scientific document QA datasets still exhibit several limitations in how supporting evidence is represented. Datasets that only provide doc/page-level evidence annotations are often too coarse-grained for scientific documents, where a single page may contain multiple dense technical elements such as paragraphs, figures, and tables. As a result, page-level supervision cannot precisely identify the specific regions that support an answer ( Ma et al., 2024 ; Li et al., 2024a ; Singh et al., 2024 ; Baumgärtner et al., 2025 ) . Furthermore, some datasets construct question answering tasks around specific structured components , such as a single table or figure ( Wang et al., 2024b ; Masry et al., 2022 ; Pramanick et al., 2024 ; Yue et al., 2025 ) . While useful for studying individual modalities, these settings isolate individual visual elements from the broader document context and therefore cannot capture the complete semantic information of the document. In contrast, token-level annotations, although more fine-grained, often fragment semantically coherent units and lose important contextual information, making systematic reasoning difficult and reducing interpretability ( Giovannini et al., 2025 ; Tanaka et al., 2021 ; Van Landeghem et al., 2023 ) . However, incorporating precise evidence region localization into the reasoning process has been shown to be effective in visual question answering ( Zheng et al., 2025 ; Hong et al., 2025 ; Zhang et al., 2025 ) , yet this capability is rarely systematically evaluated in existing scientific DocVQA benchmarks. Therefore, an effective evidence representation should support semantic region-level grounding , which preserves semantic completeness while enabling precise spatial localization of the supporting evidence. To address these limitations, we introduce SciEGQA , a visual question answering dataset for scientific documents that incorporates semantic evidence grounding . In SciEGQA, the supporting evidence for each answer is represented as semantically coherent document regions annotated with bounding boxes. This design provides an intermediate granularity between page-level and token-level annotations, enabling precise spatial localization while preserving semantic completeness, which is more suitable for multimodal reasoning over scientific documents. SciEGQA consists of two complementary components: a human-annotated benchmark constructed from 80 scientific papers across eight research domains with 1,623 high-quality QA pairs, and a large-scale automatically constructed training set generated from more than 3.6K papers containing over 30K QA pairs. The dataset supports three reasoning scenarios with increasing complexity: Single Page Single Region (SPSR) , Single Page Multi Regions (SPMR) , and Multi Pages Multi Regions (MPMR) . We further demonstrate through supervised fine-tuning experiments that the automatically constructed training set provides effective supervision for improving scientific reasoning capability. Based on SciEGQA, we conduct comprehensive evaluations on a wide range of state-of-the-art vision–language models. Experimental results reveal that even the most advanced models still struggle with evidence localization and evidence-based reasoning in scientific documents. In particular, most models achieve less than 40\% IoU in evidence grounding tasks, highlighting the challenging nature of scientific document understanding. The main contributions of this work are summarized as follows: • We introduce SciEGQA , a new benchmark and training set for evidence grounded question answering and reasoning on scientific documents. • We propose semantic region level evidence grounding , an intermediate annotation granularity that preserves semantic completeness while enabling precise spatial localization for reasoning. • We propose a Grounding–Crop–then–Answer evaluation protocol that disentangles evidence localization from reasoning ability. 2. Related Work Figure 1 . Comparison of evidence annotation granularity in document QA datasets. Existing benchmarks provide page-level QA, token-level grounding, or component-level reasoning. In contrast, SciEGQA supports page-level question answering with semantic region grounding via bounding-box evidence. Document Visual Question Answering (DocVQA) has attracted significant attention in recent years, leading to the development of various benchmark datasets for evaluating document understanding capabilities. While many datasets focus on general document types such as receipts, forms, and infographics, scientific documents introduce additional challenges due to their domain-specific characteristics and complex multimodal structures. As illustrated in Figure 1 and Table 1 , existing Scientific DocVQA datasets can be broadly categorized into three types: doc/page-level datasets, structured component-level datasets, and token-level datasets. Doc/Page-level datasets. Early datasets such as DocVQA ( Mathew et al., 2021 ) primarily focus on question answering over single-page documents. MP-DocVQA ( Tito et al., 2023 ) extends this setting to multi-page scenarios, introducing challenges such as cross-page reasoning and long-context understanding. In addition, datasets including SciDQA ( Singh et al., 2024 ) , PeerQA ( Baumgärtner et al., 2025 ) and TAT-DQA ( Zhu et al., 2022 ) further explore question answering over scientific paper-style documents. More recently, benchmarks such as MMLongBench-Doc ( Ma et al., 2024 ) have been proposed to systematically evaluate long-document understanding capabilities of multimodal models. However, these datasets typically provide only page-level evidence annotations, lacking precise localization of answer-supporting regions. Structured component-level datasets. Another line of work focuses on structured visual components in documents, such as charts, tables, and figures. Representative

Read the complete text