Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning

Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier · 2025

Read Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning by Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17830.txt.

Description

Nicolas Castanet, Olivier Sigaud, and Sylvain Lamprier tackle a coverage problem in agents that learn goals from pictures. In goal-based reinforcement learning, the agent builds an inner map of its world and picks goals from that map. But the map tends to over-represent places the agent visits, so regions stay unlearned. Their method, DRAG, pushes the map toward an even spread over the state space. It pairs a variational auto-encoder with a robust training scheme, where a network up-weights states the data underplays.

The effect is an agent whose goals reach beyond its experience. On tasks such as mazes and robot control with walls, DRAG improved coverage and control. It needs no pre-training and no prior knowledge of the environment. The authors call it a fix for a failure mode: an agent can only master goals its map represents.

Catalog record

CREATORNicolas Castanet, Olivier Sigaud, Sylvain Lamprier
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17830

Browse the full library · About the Tower of Babel