TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment

Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology · 2026

Read TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment by Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10983.txt.

Description

Jiaming Li and eleven colleagues present TMPO, a diffusion-alignment method that fights reward hacking without killing variety. Reinforcement learning aligns diffusion models to tasks, but reward chasing piles onto a few high-reward paths, collapsing visual variety and amplifying shaky rewards.

TMPO swaps scalar reward chasing for trajectory-level distribution matching. A Softmax Trajectory Balance goal aligns policy odds across paths with a reward-shaped Boltzmann distribution, keeping cover over all good paths as forward KL divergence does. Dynamic stochastic tree sampling, sharing path segments, cuts multi-path training time on large flow-matching models. Diffusion teams get alignment that preserves cover over all acceptable outputs.

Catalog record

CREATORJiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma, [0.3em] Huazhong University of Science, Technology
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77942
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10983

Browse the full library · About the Tower of Babel