IP-Dialog : Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data
Bo Peng, Zhiheng Wang, Heyang Gong, Chaochao Lu · 2025
Read IP-Dialog : Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data by Bo Peng, Zhiheng Wang, Heyang Gong, Chaochao Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02449.txt.
Description
Bo Peng and three co-authors evaluate a subtle social skill in twenty twenty-five. Good dialogue systems infer user backgrounds implicitly from conversation. They then personalize help without being told the facts outright. Data to test that skill is scarce, and building it by hand is costly and privacy-sensitive.
Their benchmark, IP-Dialog, uses automatic synthetic data instead. It covers ten tasks and twelve types of user attributes. An evaluation framework with four metrics scores attribute awareness and reasoning. Five causal graphs map how models reason their way to personalization. Extensive experiments yield observations about where models succeed and fail. Readers get a privacy-safe way to grade a capability that matters for helpful assistants.
Catalog record
| CREATOR | Bo Peng, Zhiheng Wang, Heyang Gong, Chaochao Lu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02449