When Is Compositional Reasoning Learnable from Verifiable Rewards?

Daniel Barzilai, Yotam Wolf, Ronen Basri · 2026

Read When Is Compositional Reasoning Learnable from Verifiable Rewards? by Daniel Barzilai, Yotam Wolf, Ronen Basri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07992.txt.

Description

Reinforcement learning with verifiable rewards has unlocked striking compositional reasoning in language models, but theory has lagged behind: which compositional problems can outcome-only feedback actually teach? Daniel Barzilai and two co-authors identify the task-advantage ratio, a joint property of the problem and the base model, as the key quantity.

When correct intermediate steps give a clear advantage, compositional problems are efficiently learnable from outcome feedback, and the authors analyze how such advantage arises in different problems. Without it, training may converge to suboptimal compositions, and base-model quality can become the limiting factor. The paper gives a principled map of when reward-only training suffices for compositional reasoning and when it cannot.

Catalog record

CREATORDaniel Barzilai, Yotam Wolf, Ronen Basri
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07992

Browse the full library · About the Tower of Babel