Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems

Yilun Zhao Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Jinbiao Wei Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Tingyu Song Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Siyue Zhang S \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}}, Arman Cohan Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, 𝑺 \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}} National University of Singapore, 𝑵 \hskip 1.00006pt{}^{{\color[rgb]{0.3398,0.0234,0.5508}\bm{N}}} NYU Shanghai · 2026

Read Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems by Yilun Zhao Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Jinbiao Wei Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Tingyu Song Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Siyue Zhang S \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}}, Arman Cohan Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, 𝑺 \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}} National University of Singapore, 𝑵 \hskip 1.00006pt{}^{{\color[rgb]{0.3398,0.0234,0.5508}\bm{N}}} NYU Shanghai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04018.txt.

Description

Yilun Zhao and five co-authors rethink reasoning-intensive retrieval, the task of surfacing evidence that supports downstream reasoning rather than merely matching topical similarity. Agentic search systems need retrievers that supply complementary evidence across iterative search rounds. But current benchmarks use narrow gold sets and test retrievers in isolation. Synthetic training corpora often reward single-passage relevance. The authors address both evaluation and training.

They introduce BRIGHT-Pro, an expert-annotated benchmark expanding each query with multi-aspect gold evidence, evaluated under static and agentic protocols. They also build RTriever-Synth, an aspect-decomposed synthetic corpus with matching positives and hard negatives, and use it to fine-tune RTriever-4B. Experiments show aspect-aware and agentic evaluation expose behaviors hidden by standard metrics, and the fine-tuned retriever improves a lot over its base model.

Catalog record

CREATORYilun Zhao Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Jinbiao Wei Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Tingyu Song Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, Siyue Zhang S \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}}, Arman Cohan Y \hskip 1.00006pt{}^{{\color[rgb]{0,0.207,0.418}\bm{Y}}}, 𝑺 \hskip 1.00006pt{}^{{\color[rgb]{0.9375,0.4844,0}\bm{S}}} National University of Singapore, 𝑵 \hskip 1.00006pt{}^{{\color[rgb]{0.3398,0.0234,0.5508}\bm{N}}} NYU Shanghai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.04018

Browse the full library · About the Tower of Babel