Multimodal Tabular Reasoning with Privileged Structured Information

Jun-Peng Jiang Yu Xia Hai-Long Sun Shiyin Lu Qing-Guo Chen, Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye · 2025

Read Multimodal Tabular Reasoning with Privileged Structured Information by Jun-Peng Jiang Yu Xia Hai-Long Sun Shiyin Lu Qing-Guo Chen, Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04088.txt.

Description

Reasoning over tables — pulling facts from rows and columns and chaining them into an answer — is easy for language models when the table arrives as clean text. In real life, tables usually arrive as images. Jun-Peng Jiang and co-authors tackle that gap in 2025 with TURBO, a framework that trains a vision-language model using the structured table as privileged information available only during training.

A generator built on a strong reasoning model produces step-by-step traces that bridge the image and its structure, creating high-quality training data. The model then repeatedly generates candidate reasoning paths and keeps the advantageous ones, sharpening its skill. With only about 9,000 training examples, TURBO beat the previous best system by 7.2 percent across several datasets. The authors argue that careful bridging data, not scale alone, is what transfers structured reasoning to images of tables.

Catalog record

CREATORJun-Peng Jiang Yu Xia Hai-Long Sun Shiyin Lu Qing-Guo Chen, Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04088

Browse the full library · About the Tower of Babel