Verified Critical Step Optimization for LLM Agents

Mukai Li, Qingcheng Zeng, Tianqing Fang, Zhenwen Liang, Linfeng Song, Qi Liu, Haitao Mi, Dong Yu · 2026

Read Verified Critical Step Optimization for LLM Agents by Mukai Li, Qingcheng Zeng, Tianqing Fang, Zhenwen Liang, Linfeng Song, Qi Liu, Haitao Mi, Dong Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03412.txt.

Description

Training a language model agent on long tasks is hard because a single final reward cannot say which step mattered. Mukai Li and 7 co-authors propose Critical Step Optimization in a 2026 paper. The method hunts for critical steps, decision points where a different action demonstrably flips the task from failure to success.

Starting from the agent's own failed runs, a process reward model flags candidate steps, and an expert model suggests better actions. Only alternatives that the agent itself can carry through to a correct finish are kept as training data. On two agent benchmarks, the method improves over supervised fine-tuning by wide margins while using verified supervision at only a small share of steps.

Catalog record

CREATORMukai Li, Qingcheng Zeng, Tianqing Fang, Zhenwen Liang, Linfeng Song, Qi Liu, Haitao Mi, Dong Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03412

Browse the full library · About the Tower of Babel