SimuDICE: Offline Policy Optimization Through World Model Updates and DICE Estimation

Catalin E. Brita, Stephan Bongers, Frans A. Oliehoek · 2024

Read SimuDICE: Offline Policy Optimization Through World Model Updates and DICE Estimation by Catalin E. Brita, Stephan Bongers, Frans A. Oliehoek (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06486.txt.

Description

Catalin Brita, Stephan Bongers, and Frans Oliehoek present SimuDICE, a 2024 method for learning from logged experience without live trial and error. In offline learning, an agent must improve its plan from a fixed set of past actions. The plan it wants may drift from the plan that logged the data, which makes learning hard, and data are often scarce.

SimuDICE learns a model of the world and makes fresh simulated experience from it. It then weights those samples by how often such states and actions occur and by how much the model can be trusted. The loop steadily improves the plan. Tests matched rival methods while using fewer logged cases and fewer planning steps.

Catalog record

CREATORCatalin E. Brita, Stephan Bongers, Frans A. Oliehoek
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06486

Browse the full library · About the Tower of Babel