Optimistic critics can empower small actors
Olya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro · 2025
Read Optimistic critics can empower small actors by Olya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01016.txt.
Description
Olya Mastikhina, Dhruv Sreenivas, and Pablo Samuel Castro study small actors with hopeful critics in a 2025 paper. Actor-critic methods drive much of deep reinforcement learning. Most teams give the actor and critic the same size. Small actors have been praised, but proof was thin.
Broad tests show small actors often hurt scores in practice. Their critics tend to judge states too low in value. That low view leads to poor data collection in training. The critic thus plays a key role in the failure. Fixes that lift the low value view helped in tests. The authors say the path is now open for lean actor designs.
Catalog record
| CREATOR | Olya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01016