Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning

Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye · 2026

Read Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning by Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07441.txt.

Description

Offline reinforcement learning regularizes actor-critic methods with behavior cloning to stay near the dataset. That imitation imposes a ceiling: when dataset actions are suboptimal, the actor cannot fully exploit better actions its value function suggests. Jinzong Dong and seven co-authors analyze that ceiling formally and verify it on a controlled bandit task.

Their fix, proximal action replacement, is a plug-and-play training sample replacer that substitutes suboptimal dataset actions, letting the actor climb past the imitation ceiling, especially late in training when imitation otherwise dominates. Offline RL users get a small, principled change that removes a structural limit of a popular method.

Catalog record

CREATORJinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07441

Browse the full library · About the Tower of Babel