Optimistic critics can empower small actors

Olya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro · 2025

Read Optimistic critics can empower small actors by Olya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01016.txt.

Description

Olya Mastikhina, Dhruv Sreenivas, and Pablo Samuel Castro study small actors with hopeful critics in a 2025 paper. Actor-critic methods drive much of deep reinforcement learning. Most teams give the actor and critic the same size. Small actors have been praised, but proof was thin.

Broad tests show small actors often hurt scores in practice. Their critics tend to judge states too low in value. That low view leads to poor data collection in training. The critic thus plays a key role in the failure. Fixes that lift the low value view helped in tests. The authors say the path is now open for lean actor designs.

Catalog record

CREATOROlya Mastikhina, Dhruv Sreenivas, Pablo Samuel Castro
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01016

Browse the full library · About the Tower of Babel