Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning
Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye · 2026
Read Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning by Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07441.txt.
Description
Offline reinforcement learning regularizes actor-critic methods with behavior cloning to stay near the dataset. That imitation imposes a ceiling: when dataset actions are suboptimal, the actor cannot fully exploit better actions its value function suggests. Jinzong Dong and seven co-authors analyze that ceiling formally and verify it on a controlled bandit task.
Their fix, proximal action replacement, is a plug-and-play training sample replacer that substitutes suboptimal dataset actions, letting the actor climb past the imitation ceiling, especially late in training when imitation otherwise dominates. Offline RL users get a small, principled change that removes a structural limit of a popular method.
Catalog record
| CREATOR | Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07441