GISA: A Benchmark for General Information Seeking Assistant

Yutao Zhu 1 , † , {}^{1,\dagger,\text{\faIcon[regular]{envelope}}}, Xingshuo Zhang 1,†, Maosen Zhang 1,†, Jiajie Jin 1, Liancheng Zhang 1, Xiaoshuai Song 1, Kangzhi Zhao 2, Wencong Zeng 2, Ruiming Tang 2, Han Li 2, Ji-Rong Wen 1, Zhicheng Dou 1 , {}^{1,\text{\faIcon[regular]{envelope}}} · 2026

Read GISA: A Benchmark for General Information Seeking Assistant by Yutao Zhu 1 , † , {}^{1,\dagger,\text{\faIcon[regular]{envelope}}}, Xingshuo Zhang 1,†, Maosen Zhang 1,†, Jiajie Jin 1, Liancheng Zhang 1, Xiaoshuai Song 1, Kangzhi Zhao 2, Wencong Zeng 2, Ruiming Tang 2, Han Li 2, Ji-Rong Wen 1, Zhicheng Dou 1 , {}^{1,\text{\faIcon[regular]{envelope}}} (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08543.txt.

Description

Search agents are usually tested on queries worked backward from answers, which feel nothing like real information needs. Yutao Zhu and eleven co-authors build GISA, a benchmark of 373 human-crafted queries reflecting authentic seeking behavior. Their arXiv paper covers both finding specific facts and gathering broad information in unified tasks.

Four structured answer formats allow exact evaluation, and a live subset with periodically refreshed answers resists memorization. Complete human search trajectories accompany every query, giving gold references for process supervision. Even the best model scores only 19.30 percent exact match, collapsing further on planning-heavy tasks. Readers get a hard, realistic gauge for general information-seeking assistants.

Catalog record

CREATORYutao Zhu 1 , † , {}^{1,\dagger,\text{\faIcon[regular]{envelope}}}, Xingshuo Zhang 1,†, Maosen Zhang 1,†, Jiajie Jin 1, Liancheng Zhang 1, Xiaoshuai Song 1, Kangzhi Zhao 2, Wencong Zeng 2, Ruiming Tang 2, Han Li 2, Ji-Rong Wen 1, Zhicheng Dou 1 , {}^{1,\text{\faIcon[regular]{envelope}}}
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08543

Browse the full library · About the Tower of Babel