SimpleTIR : End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning
Zhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1 · 2025
Read SimpleTIR : End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning by Zhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02479.txt.
Description
Zhenghai Xue and six co-authors wrote a 2025 machine learning paper on training language models to reason with external tools over several turns. Reinforcement learning for such tool use often collapses because tool feedback shifts the data distribution mid-training.
The team traces the failure to void turns, steps that produce neither code nor a final answer, which trigger harmful gradient spikes. Their SimpleTIR algorithm simply filters out trajectories containing void turns before each update. That one change stabilises learning without extra machinery. Starting from a base model, scores on the AIME24 maths benchmark more than doubled over a text-only baseline. Freed from supervised fine-tuning limits, the model also found richer strategies, including cross-checking its own work and correcting itself.
Catalog record
| CREATOR | Zhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-02479