RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
Zihan Wang* 1 , Kangrui Wang* 1 , Qineng Wang* 1 , Pingyue Zhang* 1 , Linjie Li* 2 , Zhengyuan Yang 4 , Xing Jin 6 , Kefan Yu 1 , Minh Nhat Nguyen 7 , Licheng Liu 1 , Eli Gottlieb 1 , Yiping Lu 1 , Kyunghyun Cho 5 , Jiajun Wu 3 , Li Fei-Fei 3 , Lijuan Wang 4 , Yejin Choi 3 , Manling Li 1 1 Northwest · 2025
Read RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning by Zihan Wang* 1 , Kangrui Wang* 1 , Qineng Wang* 1 , Pingyue Zhang* 1 , Linjie Li* 2 , Zhengyuan Yang 4 , Xing Jin 6 , Kefan Yu 1 , Minh Nhat Nguyen 7 , Licheng Liu 1 , Eli Gottlieb 1 , Yiping Lu 1 , Kyunghyun Cho 5 , Jiajun Wu 3 , Li Fei-Fei 3 , Lijuan Wang 4 , Yejin Choi 3 , Manling Li 1 1 Northwest (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20073.txt.
Description
Wang and co-authors study how language model agents can improve themselves through multi-turn reinforcement learning. Their 2025 paper lays out a training framework and a modular system, RAGEN, for building and testing such agents. Long horizons and random feedback make agent training tricky.
Trials in four stylized worlds surface three lessons. Training shows an Echo Trap pattern of reward cliffs and gradient spikes, which a stabilized variant counters with trajectory filtering, a critic, and gradient control. Rollout design matters: varied starts, medium step size, and frequent sampling help. Fine, reasoning-aware rewards prove key, since without them agents fall back on shallow tricks or fake thoughts. Code and environments are public, the authors note.
Catalog record
| CREATOR | Zihan Wang* 1 , Kangrui Wang* 1 , Qineng Wang* 1 , Pingyue Zhang* 1 , Linjie Li* 2 , Zhengyuan Yang 4 , Xing Jin 6 , Kefan Yu 1 , Minh Nhat Nguyen 7 , Licheng Liu 1 , Eli Gottlieb 1 , Yiping Lu 1 , Kyunghyun Cho 5 , Jiajun Wu 3 , Li Fei-Fei 3 , Lijuan Wang 4 , Yejin Choi 3 , Manling Li 1 1 Northwest |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-20073