Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
Gugan Thoppe, L.A. Prashanth, Ankur Naskar, Sanjay Bhat · 2026
Read Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs by Gugan Thoppe, L.A. Prashanth, Ankur Naskar, Sanjay Bhat (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08053.txt.
Description
Gugan Thoppe, L. A. Prashanth, Ankur Naskar, and Sanjay Bhat develop reinforcement learning algorithms for exponential utility, a goal that weighs outcomes by risk aversion. Work on discounted decision processes has lacked principled value-based methods for this fixed risk-aversion setting.
Starting from a Bellman-type equation studied by Porteus, the authors derive two Q-value-style updates and show their operators contract under suitable metrics. The greedy policy at the fixed point is optimal among stationary policies. One algorithm runs on two timescales and comes with almost-sure convergence and finite-time rates; a second, one-timescale version follows a sublinear power-law operator.
Catalog record
| CREATOR | Gugan Thoppe, L.A. Prashanth, Ankur Naskar, Sanjay Bhat |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08053