Offline Policy Optimization with Posterior Sampling

Hongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang · 2026

Read Offline Policy Optimization with Posterior Sampling by Hongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07393.txt.

Description

Hongqiang Lin and five co-authors tune robot policies from fixed offline data without the excessive pessimism that usually guards against unknown ground. Model-based offline learning must avoid using model errors outside the data, yet heavy pessimism sacrifices reach. Their PSPO method treats the dynamics model as a random variable drawn from a posterior, not a single estimate.

That framing allows careful probing of new regions that still fit real dynamics. Trading off posterior and policy updates gives a tuned method that provably settles. On standard benchmarks PSPO beat leading rivals, and checks confirm it stays robust without pessimism, with each part helping in removal tests.

Catalog record

CREATORHongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07393

Browse the full library · About the Tower of Babel