MAPO: Mixed Advantage Policy Optimization for Long-Horizon Multi-Turn Dialogue

Naifan Zhang 1,2 Ruihan Sun 1 Jinwei Su 1,3 Hengjie Yang 1 Zhengyuan Pan 1,4 Zhaohan Chen 1 Xiaofan Zhang 1 1 NatureSelect 2 Tsinghua University 3 South China Normal University 4 Xiamen University · 2026

Read MAPO: Mixed Advantage Policy Optimization for Long-Horizon Multi-Turn Dialogue by Naifan Zhang 1,2 Ruihan Sun 1 Jinwei Su 1,3 Hengjie Yang 1 Zhengyuan Pan 1,4 Zhaohan Chen 1 Xiaofan Zhang 1 1 NatureSelect 2 Tsinghua University 3 South China Normal University 4 Xiamen University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06194.txt.

Description

Zhang and colleagues propose MICA for multi turn emotional support dialogue: a critic free RL framework tackling sparse rewards and weak credit assignment. Responses shape user states, so matched state comparison is unavailable and trajectory supervision falls short. MICA draws immediate and delayed credit from one potential function over the user's support state, combining an incremental distance reward with its Monte Carlo return into a mixed advantage. On EMPA, EQ-Bench, and EmoBench, MICA beats GRPO and REINFORCE++, gaining up to 42.5 on EMPA with no added rollout cost.

Catalog record

CREATORNaifan Zhang 1,2 Ruihan Sun 1 Jinwei Su 1,3 Hengjie Yang 1 Zhengyuan Pan 1,4 Zhaohan Chen 1 Xiaofan Zhang 1 1 NatureSelect 2 Tsinghua University 3 South China Normal University 4 Xiamen University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40926
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06194

Browse the full library · About the Tower of Babel