When Is Compositional Reasoning Learnable from Verifiable Rewards?
Daniel Barzilai, Yotam Wolf, Ronen Basri · 2026
Read When Is Compositional Reasoning Learnable from Verifiable Rewards? by Daniel Barzilai, Yotam Wolf, Ronen Basri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07992.txt.
Description
Reinforcement learning with verifiable rewards has unlocked striking compositional reasoning in language models, but theory has lagged behind: which compositional problems can outcome-only feedback actually teach? Daniel Barzilai and two co-authors identify the task-advantage ratio, a joint property of the problem and the base model, as the key quantity.
When correct intermediate steps give a clear advantage, compositional problems are efficiently learnable from outcome feedback, and the authors analyze how such advantage arises in different problems. Without it, training may converge to suboptimal compositions, and base-model quality can become the limiting factor. The paper gives a principled map of when reward-only training suffices for compositional reasoning and when it cannot.
Catalog record
| CREATOR | Daniel Barzilai, Yotam Wolf, Ronen Basri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07992