SimpleTIR : End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning

Zhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1 · 2025

Read SimpleTIR : End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning by Zhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02479.txt.

Description

Zhenghai Xue and six co-authors wrote a 2025 machine learning paper on training language models to reason with external tools over several turns. Reinforcement learning for such tool use often collapses because tool feedback shifts the data distribution mid-training.

The team traces the failure to void turns, steps that produce neither code nor a final answer, which trigger harmful gradient spikes. Their SimpleTIR algorithm simply filters out trajectories containing void turns before each update. That one change stabilises learning without extra machinery. Starting from a base model, scores on the AIME24 maths benchmark more than doubled over a text-only baseline. Freed from supervised fine-tuning limits, the model also found richer strategies, including cross-checking its own work and correcting itself.

Catalog record

CREATORZhenghai Xue 1∗ , Longtao Zheng 1∗ , Qian Liu 2† , Yingru Li 2 , Xiaosen Zheng 2 , Zejun Ma 2 , Bo An 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-02479

Browse the full library · About the Tower of Babel