SD-Search: On-Policy Hindsight Self-Distillation for Search-Augmented Reasoning
Yufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou · 2026
Read SD-Search: On-Policy Hindsight Self-Distillation for Search-Augmented Reasoning by Yufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18299.txt.
Description
Yufei Ma's 2026 paper presents SD-Search, giving search agents step-level credit without an external teacher. Under outcome-only learning, every search choice in a rollout shares one trajectory reward, so single queries earn no signal. SD-Search lets one model play two roles split only by conditioning. A student sees just inference-time context, while a teacher also sees a hindsight block summarizing past rollouts and outcomes, marking decisions worth making. The student matches it at search-query positions, layering dense step-level signal over trajectory reward.
Catalog record
| CREATOR | Yufei Ma, Zihan Liang, Ben Chen, Zhipeng Qian, Huangyu Dai, Xuxin Zhang, Chenyi Lei, Wenwu Ou |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65856 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18299