Agentic Transformers Provably Learn to Search via Reinforcement Learning
Tong Yang, Yu Huang, Yingbin Liang, Yuejie Chi · 2026
Read Agentic Transformers Provably Learn to Search via Reinforcement Learning by Tong Yang, Yu Huang, Yingbin Liang, Yuejie Chi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00183.txt.
Introduction
Yang and co-authors ask how a transformer learns to search a tree by trial and reward. Tree search sits at the heart of many agent tasks. Agents must try moves, note fails, and step back to bright paths. How nets gain this skill from sparse reward was not clear.
The team first hand-builds a two-head net that does depth-first search. One head tracks past moves, and one head spots fails and calls for back steps. They then show the same split grows on its own in training. A step-wise plan feeds depth one, then depth two trees. The trained net still cracks much deeper trees. With skewed goals, a tweak makes it rank bright limbs first.
Catalog record
| CREATOR | Tong Yang, Yu Huang, Yingbin Liang, Yuejie Chi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 394 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00183