SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables

Sungho Park, Jueun Kim, Wook-Shin Han · 2026

Read SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables by Sungho Park, Jueun Kim, Wook-Shin Han (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23286.txt.

Description

Sungho Park and two co-authors present SPARTA, a framework that builds large Table-Text question-answering benchmarks automatically. Real tasks demand reasoning across long text and source tables, over several hops, with operations such as aggregation. Yet existing benchmarks are small, hand-built and thus error-prone, and their questions rarely exceed two hops or call for grouping and other advanced operations.

SPARTA enriches each source table with grounding tables of atomic facts drawn from the text, then synthesizes nested queries whose predicate count matches the target hop count. Provenance-based refinement rewrites queries that return empty results, while realistic-structure enforcement limits generation to post-order traversals of the query graph. The pipeline yields thousands of quality pairs at a quarter of HybridQA's annotation time. Leading models that score over 70 F1 on HybridQA or over 50 on OTT-QA drop by more than 30 F1 points on SPARTA.

Catalog record

CREATORSungho Park, Jueun Kim, Wook-Shin Han
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23286

Browse the full library · About the Tower of Babel