What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
Xiaozhe Li *, Tianyi Lyu *, Yang Li, Yichuan Ma, Peiji Li, Linyang Li †, Qipeng Guo †, Dahua Lin, Kai Chen · 2026
Read What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents by Xiaozhe Li *, Tianyi Lyu *, Yang Li, Yichuan Ma, Peiji Li, Linyang Li †, Qipeng Guo †, Dahua Lin, Kai Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19447.txt.
Description
Li and team study how to train multi-turn agents from sparse rewards, where one success-or-failure signal must spread across many actions. They test five feedback sources at two insertion granularities and propose SERL, which lets the task reward set update direction while environment feedback tunes placement and size, focusing on critical actions. On ALFWorld and WebShop it reaches 90.0 percent and 80.1 percent success, beating strong reinforcement and distillation baselines. Grounded feedback at meaningful points beats longer, richer context used blindly.
Catalog record
| CREATOR | Xiaozhe Li *, Tianyi Lyu *, Yang Li, Yichuan Ma, Peiji Li, Linyang Li †, Qipeng Guo †, Dahua Lin, Kai Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46324 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19447