Offline Policy Optimization with Posterior Sampling
Hongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang · 2026
Read Offline Policy Optimization with Posterior Sampling by Hongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07393.txt.
Description
Hongqiang Lin and five co-authors tune robot policies from fixed offline data without the excessive pessimism that usually guards against unknown ground. Model-based offline learning must avoid using model errors outside the data, yet heavy pessimism sacrifices reach. Their PSPO method treats the dynamics model as a random variable drawn from a posterior, not a single estimate.
That framing allows careful probing of new regions that still fit real dynamics. Trading off posterior and policy updates gives a tuned method that provably settles. On standard benchmarks PSPO beat leading rivals, and checks confirm it stays robust without pessimism, with each part helping in removal tests.
Catalog record
| CREATOR | Hongqiang Lin, Dongxu Zhang, Yiding Sun, Mingzhe Li, Ning Yang, Haijun Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07393