Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization
Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma · 2026
Read Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization by Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07764.txt.
Description
Agents that must balance competing goals, such as speed against safety, often collapse onto a narrow set of behaviors. Tanmay Ambadkar and four co-authors diagnose two causes: early merging of objectives cancels useful learning signals, and policies lose diversity across different preferences.
Their framework, D3PO, keeps each objective's signal separate until after trust-region stabilization, then weights preferences late, while a diversity term pushes behaviors apart in proportion to preference distance. The result recovers fuller Pareto fronts, the set of trade-offs where no goal improves without harming another. Multi-objective reinforcement learning researchers get both a diagnosis of a common failure and a fix that stays within efficient standard practice.
Catalog record
| CREATOR | Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07764