Actor-Critic Pretraining for Proximal Policy Optimization

Andreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber · 2026

Read Actor-Critic Pretraining for Proximal Policy Optimization by Andreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23804.txt.

Description

Andreas Kernbach and four co-authors present a pretraining method for actor-critic reinforcement learning, aimed at the widely used PPO algorithm. The 2026 paper notes that such learning often needs huge numbers of environment steps, which limits its use in robotics, and that expert demos can cut that cost.

Past work usually pretrains only the actor, the network that picks actions, by copying expert behavior. The new method also pretrains the critic, the network that judges actions, using returns from rollouts of the pretrained actor. Tests on fifteen simulated manipulation and locomotion tasks show large gains: sample efficiency rises sharply over no pretraining, and clearly over actor-only pretraining. Both networks, the authors argue, deserve a head start from expert data.

Catalog record

CREATORAndreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23804

Browse the full library · About the Tower of Babel