Radar : Benchmarking Language Models on Imperfect Tabular Data
Ken Gu 1,3 † \bm{\dagger} , Zhihan Zhang 1 ∗ \bm{*} , Kate Lin 1 ∗ \bm{*} , Yuwei Zhang 1 ∗ \bm{*} , Akshay Paruchuri 1 ∗ \bm{*} , Hong Yu 1 ∗ \bm{*} , Mehran Kazemi 2 , Kumar Ayush 1 , A. Ali Heydari 1 , Maxwell A. Xu 1 , Girish Narayanswamy 1 , Yun Liu 1 , Ming-Zher Poh 1 , Yuzhe Yang 1 , Mark Mal · 2025
Read Radar : Benchmarking Language Models on Imperfect Tabular Data by Ken Gu 1,3 † \bm{\dagger} , Zhihan Zhang 1 ∗ \bm{*} , Kate Lin 1 ∗ \bm{*} , Yuwei Zhang 1 ∗ \bm{*} , Akshay Paruchuri 1 ∗ \bm{*} , Hong Yu 1 ∗ \bm{*} , Mehran Kazemi 2 , Kumar Ayush 1 , A. Ali Heydari 1 , Maxwell A. Xu 1 , Girish Narayanswamy 1 , Yun Liu 1 , Ming-Zher Poh 1 , Yuzhe Yang 1 , Mark Mal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08249.txt.
Description
Ken Gu and a large team built RADAR in 2025, a benchmark that tests how language models handle imperfect tabular data. Models now run data analyses on their own, yet real tables hold missing values, outliers, and logical clashes that can wreck conclusions if mishandled. Data awareness of this kind was little tested.
RADAR holds close to three thousand table-query pairs from real data across nine domains, with five artifact types added by program. Table size is varied to test how reasoning holds as tables grow. Top models did decently on clean tables but degraded sharply once artifacts appeared. The benchmark is built to extend to new flaw types and sizes.
Catalog record
| CREATOR | Ken Gu 1,3 † \bm{\dagger} , Zhihan Zhang 1 ∗ \bm{*} , Kate Lin 1 ∗ \bm{*} , Yuwei Zhang 1 ∗ \bm{*} , Akshay Paruchuri 1 ∗ \bm{*} , Hong Yu 1 ∗ \bm{*} , Mehran Kazemi 2 , Kumar Ayush 1 , A. Ali Heydari 1 , Maxwell A. Xu 1 , Girish Narayanswamy 1 , Yun Liu 1 , Ming-Zher Poh 1 , Yuzhe Yang 1 , Mark Mal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 147 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08249