Actor-Critic Pretraining for Proximal Policy Optimization
Andreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber · 2026
Read Actor-Critic Pretraining for Proximal Policy Optimization by Andreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23804.txt.
Description
Andreas Kernbach and four co-authors present a pretraining method for actor-critic reinforcement learning, aimed at the widely used PPO algorithm. The 2026 paper notes that such learning often needs huge numbers of environment steps, which limits its use in robotics, and that expert demos can cut that cost.
Past work usually pretrains only the actor, the network that picks actions, by copying expert behavior. The new method also pretrains the critic, the network that judges actions, using returns from rollouts of the pretrained actor. Tests on fifteen simulated manipulation and locomotion tasks show large gains: sample efficiency rises sharply over no pretraining, and clearly over actor-only pretraining. Both networks, the authors argue, deserve a head start from expert data.
Catalog record
| CREATOR | Andreas Kernbach, Amr Elsheikh, Nicolas Grupp, René Nagel, and Marco F. Huber |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23804