Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation

Jiani Luo, Xiaoyan Zhao, Yang Zhang, Shuyi Miao, Bingbing Xu, Stefan Konigorski, Tat-Seng Chua · 2026

Read Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation by Jiani Luo, Xiaoyan Zhao, Yang Zhang, Shuyi Miao, Bingbing Xu, Stefan Konigorski, Tat-Seng Chua (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24647.txt.

Introduction

Jiani Luo and others argue personalizing dialogue takes more than recalling history. Systems must infer hidden states that shift as conversation unfolds. Their PUMA framework treats personalization as choice under partial observability, grounded in the free-energy principle. Each turn it holds a belief about the latent state. It updates a model of observations and action-driven state changes. It picks actions minimizing free energy, balancing learning about the user against serving the user. Counseling and motivational-interviewing tests show stronger long-horizon outcomes with reliable state tracking.

Catalog record

CREATORJiani Luo, Xiaoyan Zhao, Yang Zhang, Shuyi Miao, Bingbing Xu, Stefan Konigorski, Tat-Seng Chua
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88434
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24647

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation Jiani Luo, Xiaoyan Zhao, Yang Zhang, Shuyi Miao, Bingbing Xu, Stefan Konigorski, Tat-Seng Chua Abstract Personalized dialogue requires more than recalling explicit user histories: systems also need to infer hidden user states that evolve through interaction and shape appropriate response strategies. Existing memory- and profile-based methods primarily reuse observable user information, offering limited support for modeling user-state dynamics or selecting actions based on how they shape future user states. We propose PUMA (Prospective User-state Modeling for Action selection), a framework grounded in the Free Energy Principle (FEP) that formulates personalization as decision-making under partial observability, centered on an explicit user state model that captures latent user states and their action-conditioned dynamics. At each turn, PUMA maintains a belief over the user’s hidden state, refines the user state model for observation generation and action-conditioned state transition, and selects dialogue actions by minimizing expected free energy—balancing epistemic and pragmatic objectives under a unified criterion. This formulation shifts personalization from passive memory retrieval to model-based decision-making over user evolution. We instantiate PUMA on healthcare-oriented counseling and motivational interviewing benchmarks with latent state annotations for rigorous evaluation. Experiments show that PUMA improves long-horizon dialogue outcomes while maintaining strong response quality, and a cross-dataset study demonstrates more reliable user-state estimation and next-state prediction. Our code is available at: https://github.com/Annie1161/PUMA . 1 Introduction Personalized dialogue systems aim to generate responses that remain coherent, adaptive, and user-specific over long-term interactions [ 55 , 16 , 2 ] . They are important in applications such as personal assistants, education, healthcare communication, mental health support, and long-term decision support, where effective interaction depends on understanding both the current query and the user’s evolving context. Recent LLM-based systems [ 56 , 27 , 54 , 37 , 51 , 44 ] have advanced personalization by incorporating long-term memory, user profiles, and retrieved interaction histories, enabling models to reuse explicit user information such as preferences, decisions, and prior dialogue content. However, long-term personalization cannot be reduced to recalling explicitly stated user information. The same utterance may require fundamentally different responses depending on the user’s latent state [ 3 ] . Consider a user who says, "I’ve been taking the new medication for two weeks". The appropriate response depends on whether the user is anxious about side effects, confidently tracking progress, or subtly reconsidering discontinuation. These differences reflect latent factors such as uncertainty, engagement, and readiness to act [ 14 ] , which are only partially expressed in language but critically shape how an utterance should be interpreted and how the system should respond [ 3 , 35 , 19 ] . Therefore, effective personalization requires not only tracking what the user has said, but also inferring hidden user states and reasoning about how system actions influence their evolution over time. This points to the need for a user state model—a predictive representation of the user as a latent dynamical system whose state evolves under system interventions—as the foundation for long-term personalization. Existing personalized dialogue systems remain limited from this perspective. Memory-augmented and profile-based methods [ 53 , 13 , 47 , 42 ] operate over explicit textual records, making them effective for recalling stable facts but insufficient for modeling latent state evolution. While some approaches introduce latent variables, these representations are typically static or descriptive, failing to capture the continuous and action-dependent dynamics of user behavior [ 39 , 27 , 17 ] . As a result, existing systems implicitly treat personalization as a retrieval or conditioning problem, rather than a sequential decision-making problem over a partially observable user. In short, current methods either lack a user state model entirely or rely on impoverished, non-predictive surrogates—leaving prospective reasoning about user evolution under system actions out of reach. The Free Energy Principle (FEP) and active inference [ 8 , 5 ] provide a principled decision-theoretic foundation for addressing this gap. Under FEP, an agent operates in a partially observable environment by maintaining and updating beliefs over hidden states, and selecting actions by minimizing expected free energy (EFE), which balances uncertainty reduction (epistemic value) and goal-directed behavior (pragmatic value). Since EFE is evaluated over action-conditioned dynamics, actions are chosen based on both their immediate effects and their influence on future state beliefs. This formulation naturally aligns with personalized dialogue, where the user is a latent dynamical system and system responses act as interventions that shape its evolution. Building on this perspective, we propose PUMA ( P rospective U ser-state M odeling for A ction selection), an FEP-grounded framework for personalized dialogue. At the core of PUMA lies a user state model, defined as a predictive model over latent user states and their action-conditioned dynamics, capturing how the user evolves in response to system interventions. This goes beyond static state estimation by explicitly modeling state transitions induced by dialogue actions. At each dialogue turn, PUMA maintains a belief over the user’s latent state and a world model of user-state dynamics, updates them based on observations, and evaluates candidate responses by estimating their expected impact on future states through an approximation of expected free energy. This enables joint inference and control within a unified decision framework grounded in the FEP. This formulation shifts personalization from passive memory retrieval to model-based decision-making over user evolution. Rather than selecting responses based on retrieved context or static preferences, PUMA evaluates candidate actions by their expected influence on future user trajectories, balancing epistemic and pragmatic objectives under a single principled criterion. We instantiate PUMA on healthcare dialogue benchmarks, which provide the latent user state annotations necessary for evaluating belief tracking and long-horizon decision quality. Experimental results show that explicitly modeling user state dynamics and performing FEP-based action evaluation significantly improve both user state tracking and long-horizon dialogue performance compared with strong LLM-based baselines. Our main contributions are summarized as follows: • We introduce a user state model for personalized dialogue that, unlike static persona, memory-based or profile-based representations in prior work, captures latent user states and their action-conditioned evolution over time. • We formulate personalized dialogue under the Free Energy Principle and develop PUMA, an FEP-grounded framework that unifies belief updating and action selection via expected free energy, enabling prospective decision-making by reasoning over future user states. • We empirically evaluate PUMA on healthcare dialogue benchmarks, where FEP-grounded user state modeling and response selection yield consistent gains in both belief tracking and long-horizon dialogue outcomes over strong LLM-based baselines. 2 Related Work 2.1 Personalized Dialogue Systems Personalized dialogue systems aim to generate responses that remain coherent, adaptive, and user-specific across long-term interactions. Existing studies can be broadly categorized into profile-based personalization, memory-augmented dialogue systems, long-context personalization, and latent-variable dialogue modeling. Profile-, memory-, and long-context-based methods represent users with persona descriptions, preference profiles, structured attributes [ 56 , 31 , 55 , 38 , 22 ] , short- or long-term memories [ 48 , 13 , 47 , 49 , 57 ] , or extended interaction histories [ 32 , 1 , 15 , 36 , 29 , 50 , 30 ] , and inject them through conditioning, retrieval, or long-context prompting to reuse past interactions, stated preferences, decisions, and personal facts. In parallel, latent-variable and state-aware

Read the complete text