TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment
Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology · 2026
Read TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment by Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10983.txt.
Description
Jiaming Li and eleven colleagues present TMPO, a diffusion-alignment method that fights reward hacking without killing variety. Reinforcement learning aligns diffusion models to tasks, but reward chasing piles onto a few high-reward paths, collapsing visual variety and amplifying shaky rewards.
TMPO swaps scalar reward chasing for trajectory-level distribution matching. A Softmax Trajectory Balance goal aligns policy odds across paths with a reward-shaped Boltzmann distribution, keeping cover over all good paths as forward KL divergence does. Dynamic stochastic tree sampling, sharing path segments, cuts multi-path training time on large flow-matching models. Diffusion teams get alignment that preserves cover over all acceptable outputs.
Catalog record
| CREATOR | Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77942 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10983