Unlocking Proactivity in Task-Oriented Dialogue
Azure Zhang, Ning Gao, Yuqin Dai, Ruiyuan Wu, Jinpeng Wang, Rena Wei Gao, Bingdong Tan, Shuzheng Gao, Zongjie Li, Chaozheng Wang · 2026
Read Unlocking Proactivity in Task-Oriented Dialogue by Azure Zhang, Ning Gao, Yuqin Dai, Ruiyuan Wu, Jinpeng Wang, Rena Wei Gao, Bingdong Tan, Shuzheng Gao, Zongjie Li, Chaozheng Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22240.txt.
Introduction
Zhang and colleagues address proactive task dialogue, such as outbound sales. An agent there must probe a user's worries and steer toward acceptance within limited turns. Language models tend to stay cautious. Reward shaping only reweights what a passive policy already samples. Their answer centres on hidden concerns. A Cognitive User Simulator models each user from visible traits and hidden concerns. A training method then transfers concern-aware behaviour into the conversation-only view.
Catalog record
| CREATOR | Azure Zhang, Ning Gao, Yuqin Dai, Ruiyuan Wu, Jinpeng Wang, Rena Wei Gao, Bingdong Tan, Shuzheng Gao, Zongjie Li, Chaozheng Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39022 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22240
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unlocking Proactivity in Task-Oriented Dialogue Azure Zhang, Ning Gao, Yuqin Dai, Ruiyuan Wu, Jinpeng Wang, Rena Wei Gao, Bingdong Tan, Shuzheng Gao, Zongjie Li, Chaozheng Wang Abstract Proactive task-oriented dialogue (TOD), such as outbound sales, demands a persuasive agent that actively probes the user’s concerns and steers the conversation toward acceptance within a bounded number of turns. Yet post-trained LLMs are inherently conservative, and reward-shaping RL (e.g., GRPO) struggles since it only re-weights what an already passive policy samples. We show that conditioning on the user’s latent concerns unlocks proactive capability that no amount of sampling can undermine, establishing these concerns as a pivotal training-time signal. To operationalize this finding, we build the Cognitive User Simulator , which models each user as a stratified persona comprising observable external traits and hidden internal concerns. The simulator produces faithful and diverse interactions, while emitting per-turn state dynamics that track persuasion progress. We then introduce Simulator-Induced Asymmetric-View Policy Optimization , which converts the modeled concerns and the simulation state transition into complementary training objectives: (1) Asymmetric On-Policy Self-Distillation that transfers concern-aware behavior from a privileged view of the same policy into its deployable, conversation-only view; and (2) State-Transition Policy Refinement where the final decision provides trajectory-level advantage while synchronous state transitions refine turn-level credit direction. Across two real-world food-delivery benchmarks ( merchant and courier outbound recruitment), our method matches or surpasses leading proprietary LLMs, outperforms strong RL baselines, and generalizes across various LLM-based user simulators. 1 Introduction Large Language Models (LLMs) have demonstrated strong conversational capabilities, leading interests in applying them to task-oriented dialogue (TOD) [ 27 , 4 , 6 , 5 ] . Among TOD scenarios, proactive settings such as outbound sales are especially challenging compared with conventional reactive ones, where agents respond to users’ explicit and direct requests [ 27 ] : the agent must initiate or sustain the interaction, push the dialogue toward a predefined goal, and persuade users to take a target action [ 8 ] . These requirements impose two core demands beyond fluent response generation: agents must identify users’ inherent concerns and implicit needs and proactively steer the conversation toward acceptance . Figure 1 : Pilot study. Latent concerns move agents from the reactive plateau to the high-proactivity/high-acceptance regime, whereas sampling and GRPO provide only small shifts. However, these demands expose two structural gaps that existing approaches cannot bridge. Gap1: Current user simulators lack modeling of the implicit concerns. Existing simulators typically model only surface traits (e.g., tone, politeness) [ 32 , 10 , 8 ] , which makes simulated users differ mainly in style, causing dialogues to collapse into homogeneous trajectories with limited behavioral diversity. Crucially, such surface traits do not capture the latent concerns that actually drive user decisions, making simulator decisions weakly grounded and noisy. Gap2: LLMs are inherently reactive. Standard alignment optimizes models to respond helpfully rather than to drive the conversation, leaving them ill-suited for the initiative-taking required by proactive TOD. Mitigating this gap with current post-training formats proves equally difficult: SFT [ 34 , 11 , 7 ] lacks high-quality data, merely mimicking surface utterances; RL methods [ 29 , 19 , 10 , 8 ] struggle to credit the specific turns where persuasion occurs, whether relying on sparse outcome rewards or LLM judges that score generic dialogue quality rather than actual progress toward acceptance. These two gaps meet at the same training intersection for LLMs: the user’s latent decision process. Without modeling the concerns that guide user decisions, simulators cannot provide grounded decisions, and training methods cannot identify which proactive turns actually resolve user resistance. To further examine whether this missing variable limits current LLM policies, we conduct a pilot study to evaluate the capability and proactivity of LLMs in proactive TOD. Figure 1 summarizes the results in the (Proactivity, Acceptance Rate) plane. Vanilla models (e.g., Qwen3-4B/8B) anchor at low operating points, and test-time scaling with Pass@16 leaves them clustered nearby, suggesting that more sampling alone cannot elicit the required persuasive behaviors. Reward-shaping RL (e.g., GRPO) consequently exhibits a similar ceiling, as it can only re-weight behaviors already sampled by the policy. By contrast, conditioning the same agents on users’ latent concerns yields substantial gains in both proactivity and acceptance (dashed arrows). This confirms that the user’s latent concerns are the critical missing variable: once provided, they unlock proactive capability that no amount of sampling or reward shaping can recover, establishing latent concerns not merely as a simulator detail but as a pivotal training-time signal for learning proactive policies. Guided by this finding, we propose a co-designed framework that models latent concerns to build a faithful training environment and then converts the resulting information into complementary learning objectives. The Cognitive User Simulator (CUS) models each user as a stratified persona comprising observable external traits, hidden internal concerns \mathcal{P}^{\mathrm{int}} , and synchronous state dynamics. By explicitly modeling the latent concerns that anchor user decisions, CUS produces behaviorally diverse and faithful interactions, providing a high-fidelity training environment that surface-trait-only simulators cannot offer. It further emits simulation feedback—the final accept/reject outcome and per-turn internal state transitions—that records how user receptiveness changes in each turn. We then introduce Simulator-Induced Asymmetric-View Policy Optimization to convert the modeled concerns and the simulation state into complementary training objectives. (1) Asymmetric On-Policy Self-Distillation (AOPD) evaluates a single policy under two asymmetric information views: a privileged view that additionally conditions on \mathcal{P}^{\mathrm{int}} , and a deployable view that observes only the dialogue history. The privileged view’s concern-aware behavior is progressively distilled into the deployable one, requiring no separate teacher model and enabling the agent to implicitly infer and address latent concerns at inference time. (2) State-Transition Policy Refinement (STPR) complements this with evaluative policy-gradient signals: the final decision establishes trajectory-level advantage while per-turn state transitions refine step-level credit direction—all derived from CUS without external reward models or judges. Extensive experiments validate both components. Human evaluation confirms that CUS produces substantially more diverse and truthful dialogue topics and more grounded decisions than prior LLM simulators. Built on CUS, our trained policy achieves higher task success on real-world outbound-call benchmarks, and is remarkably more persuasive than RL baselines and current flagship LLMs, evaluated by both powerful LLM-as-judge and human evaluators. We make the following contributions: • We propose Cognitive User Simulator (CUS), which models each user as a stratified persona with hidden internal concerns \mathcal{P}^{\mathrm{int}} and synchronous state dynamics, calibrated on 100K+ real-world outbound-call logs, producing faithful and behaviorally diverse interactions. • We introduce Simulator-Induced Asymmetric-View Policy Optimization, which converts the modeled concerns and simulation state from CUS into complementary training objectives via Asymmetric On-Policy Self-Distillation and State-Transition Policy Refinement, with no external reward model, judge, or another teacher model. • Extensive experiments on two real-world food-delivery proactive-TOD benchmarks ( merchant and courier outbound recruitment) demonstrate that our method matches or surpasses leading flagship LLMs (e.g., Claude-Sonnet-4.5, GLM-5.1) and outperforms strong RL baselines, while remaining robust across other LLM-based user simulators. 2 Related Work Task-oriented dialogue (TOD) has evolved from sequence-to-sequence architectures [ 22 , 25 ] to LLM-driven paradigms that leverage in-context reasoning and instruction following [ 24 , 28 ] , with reinforcement learning increasingly applied for policy optimization in multi-turn settings [ 30 , 23 , 10