MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space

Anshul Singh, Chris Biemann, Jan Strich · 2025

Read MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space by Anshul Singh, Chris Biemann, Jan Strich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11684.txt.

Description

Anshul Singh, Chris Biemann, and Jan Strich introduce MTabVQA, a benchmark for reasoning across several tables shown as images, in a 2025 paper. Vision-language models read single tables fairly well, but real documents often spread facts over many tables on web pages and files. Tests for that combined skill were missing.

The benchmark holds 3,745 hard question-answer pairs that need steps across multiple table images. Top current models scored poorly, which exposes a clear gap. The team also built a large training set of worked examples. Tuning models on that set raised their scores on multi-table questions by a wide margin. Both the benchmark and the training data are shared in public.

Catalog record

CREATORAnshul Singh, Chris Biemann, Jan Strich
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11684

Browse the full library · About the Tower of Babel