Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning

Narjes Nourzad, Carlee Joe-Wong · 2026

Read Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning by Narjes Nourzad, Carlee Joe-Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17931.txt.

Description

A second paper by Narjes Nourzad and Carlee Joe-Wong addresses the same sparse-reward strain from a different angle. Reinforcement learning needs many interactions to learn, and language models help by finding subgoals and guiding paths. Frequent model calls, however, raise scale and trust worries.

The authors build a memory graph encoding subgoals and paths from model guidance and the agent's own successful runs. From that graph they derive a utility function scoring how closely agent paths match prior successful strategies. That utility shapes the advantage function, giving the critic extra guidance without altering rewards. The method leans mainly on offline input with only occasional live queries, avoiding dependence on constant model supervision. Early tests in benchmark settings show faster early learning and better sample speed than baseline methods, with final returns comparable to frequent-interaction methods.

Catalog record

CREATORNarjes Nourzad, Carlee Joe-Wong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE10 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17931

Browse the full library · About the Tower of Babel