FDABench: A Benchmark for Data Agents on Analytical Queries over Heterogeneous Data

Ziting Wang 1 Shize Zhang 2 Haitao Yuan 1 Jinwei Zhu 3 , Shifu Li 3 , Wei Dong 1 , Gao Cong 1 · 2025

Read FDABench: A Benchmark for Data Agents on Analytical Queries over Heterogeneous Data by Ziting Wang 1 Shize Zhang 2 Haitao Yuan 1 Jinwei Zhu 3 , Shifu Li 3 , Wei Dong 1 , Gao Cong 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02473.txt.

Description

Ziting Wang and five co-authors present a 2025 benchmark paper, FDABench, for data agents that answer analytical questions across mixed data sources. Such agents must reason over databases, documents, web pages, images, video, and audio together.

Older benchmarks test single skills or few data types, so coverage stays thin. FDABench holds about two thousand tasks spanning six data modalities, scored under one multi-level evaluation framework. Its tasks were built with PUDDING, a construction pipeline that pairs model generation with repeated expert checks for reliability. The authors run several agent designs, including general analytical agents, semantic operator frameworks, and retrieval-based methods. Comparisons reveal where each design struggles and offer guidance for building stronger data agents.

Catalog record

CREATORZiting Wang 1 Shize Zhang 2 Haitao Yuan 1 Jinwei Zhu 3 , Shifu Li 3 , Wei Dong 1 , Gao Cong 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-02473

Browse the full library · About the Tower of Babel