SD-Search: On-Policy Hindsight Self-Distillation for Search-Augmented Reasoning

Yufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou · 2026

Read SD-Search: On-Policy Hindsight Self-Distillation for Search-Augmented Reasoning by Yufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18299.txt.

Description

Yufei Ma's 2026 paper presents SD-Search, giving search agents step-level credit without an external teacher. Under outcome-only learning, every search choice in a rollout shares one trajectory reward, so single queries earn no signal. SD-Search lets one model play two roles split only by conditioning. A student sees just inference-time context, while a teacher also sees a hindsight block summarizing past rollouts and outcomes, marking decisions worth making. The student matches it at search-query positions, layering dense step-level signal over trajectory reward.

Catalog record

CREATORYufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65856
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18299

Browse the full library · About the Tower of Babel