MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space
Anshul Singh, Chris Biemann, Jan Strich · 2025
Read MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space by Anshul Singh, Chris Biemann, Jan Strich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11684.txt.
Description
Anshul Singh, Chris Biemann, and Jan Strich introduce MTabVQA, a benchmark for reasoning across several tables shown as images, in a 2025 paper. Vision-language models read single tables fairly well, but real documents often spread facts over many tables on web pages and files. Tests for that combined skill were missing.
The benchmark holds 3,745 hard question-answer pairs that need steps across multiple table images. Top current models scored poorly, which exposes a clear gap. The team also built a large training set of worked examples. Tuning models on that set raised their scores on multi-table questions by a wide margin. Both the benchmark and the training data are shared in public.
Catalog record
| CREATOR | Anshul Singh, Chris Biemann, Jan Strich |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11684