Learning from Active Human Involvement through Proxy Value Propagation

Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li †, Bolei Zhou · 2025

Read Learning from Active Human Involvement through Proxy Value Propagation by Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li †, Bolei Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03369.txt.

Description

Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li, and Bolei Zhou published this paper in 2025. It teaches AI agents by letting a human step in during training and take over when the agent goes wrong. Such guidance keeps learning safe and closer to what people want.

Their method needs no reward signal. Instead, it builds a stand-in value score that rates the human's demonstrated moves as good and the agent's corrected moves as poor. A standard learning rule then spreads those ratings to nearby states the agent met on its own. The policy that emerges copies the human's intent. Tests covered both simple control tasks and driving in a video game, with several kinds of human controls. The method worked across all of them with little change to standard code.

Catalog record

CREATORZhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li †, Bolei Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03369

Browse the full library · About the Tower of Babel