Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization

Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma · 2026

Read Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization by Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07764.txt.

Description

Agents that must balance competing goals, such as speed against safety, often collapse onto a narrow set of behaviors. Tanmay Ambadkar and four co-authors diagnose two causes: early merging of objectives cancels useful learning signals, and policies lose diversity across different preferences.

Their framework, D3PO, keeps each objective's signal separate until after trust-region stabilization, then weights preferences late, while a diversity term pushes behaviors apart in proportion to preference distance. The result recovers fuller Pareto fronts, the set of trade-offs where no goal improves without harming another. Multi-objective reinforcement learning researchers get both a diagnosis of a common failure and a fix that stays within efficient standard practice.

Catalog record

CREATORTanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07764

Browse the full library · About the Tower of Babel