Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning
Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier · 2025
Read Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning by Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17830.txt.
Description
Nicolas Castanet, Olivier Sigaud, and Sylvain Lamprier tackle a coverage problem in agents that learn goals from pictures. In goal-based reinforcement learning, the agent builds an inner map of its world and picks goals from that map. But the map tends to over-represent places the agent visits, so regions stay unlearned. Their method, DRAG, pushes the map toward an even spread over the state space. It pairs a variational auto-encoder with a robust training scheme, where a network up-weights states the data underplays.
The effect is an agent whose goals reach beyond its experience. On tasks such as mazes and robot control with walls, DRAG improved coverage and control. It needs no pre-training and no prior knowledge of the environment. The authors call it a fix for a failure mode: an agent can only master goals its map represents.
Catalog record
| CREATOR | Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17830