AI Persona :Towards Life-long Personalization of LLMs

Tiannan Wang, Meiling Tao, Ruoyu Fang, Huilin Wang, Shuai Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou · 2024

Read AI Persona :Towards Life-long Personalization of LLMs by Tiannan Wang, Meiling Tao, Ruoyu Fang, Huilin Wang, Shuai Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13103.txt.

Introduction

Tiannan Wang and co-authors argue that AI assistants should keep adapting to each user for life, and they set out the task in formal terms. Most current work chases bigger models trained on more data, while real users change jobs, tastes, and needs over time. A system that cannot track those shifts gives stale help. The team defines what lifelong personalization means, then offers a framework they describe as simple, general, effective, and able to scale.

Alongside the framework, they present ways to build realistic test sets for this task without waiting years for real data. They also propose measures for judging how well a system keeps up with a changing person. They plan to release their code and data so others can build on the setup. The paper positions steady, personal adaptation as a research direction equal in weight to raw capability.

Catalog record

CREATORTiannan Wang, Meiling Tao, Ruoyu Fang, Huilin Wang, Shuai Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-13103

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AI Persona :Towards Life-long Personalization of LLMs Tiannan Wang, Meiling Tao, Ruoyu Fang, Huilin Wang, Shuai Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou Abstract In this work, we introduce the task of life-long personalization of large language models. While recent mainstream efforts in the LLM community mainly focus on scaling data and compute for improved capabilities of LLMs, we argue that it is also very important to enable LLM systems, or language agents, to continuously adapt to the diverse and ever-changing profiles of every distinct user and provide up-to-date personalized assistance. We provide a clear task formulation and introduce a simple, general, effective, and scalable framework for life-long personalization of LLM systems and language agents. To facilitate future research on LLM personalization, we also introduce methods to synthesize realistic benchmarks and robust evaluation metrics. We will release all codes and data for building and benchmarking life-long personalized LLM systems. 1 1 1 https://github.com/tml1026/Lifelong-Personalized-Agent 1 Introduction The advent of large language models (LLMs) that can effectively understand and generate human language ( Radford et al., 2018 ; Radford et al., 2019 ; Brown et al., 2020 ; Ouyang et al., 2022 ; OpenAI, 2023 ; Touvron et al., 2023a ; Touvron et al., 2023b ) has marked a transformative era in artificial intelligence. LLMs such as ChatGPT, Gemini, and Claude have shown remarkable capabilities, not only as general chatbot providing useful responses but also as agents ( Zhou et al., 2023a ; Zhou et al., 2024 ; Chen et al., 2023 ; Liang et al., 2024b ; Liang et al., 2024a ) that assist hundreds of millions of users in diverse real-world tasks. Mainstream efforts in the AI/LLM community have been focusing on scaling data and compute on pre-training or post-training stages for building stronger LLMs that can better serve the needs of users. Millions of GPUs and billions of dollars are consumed every year for this purpose. However, a crucial question is: does satisfactory user experience naturally emerges with improved capabilities of LLMs? While it is true that improved reasoning abilities help LLMs solve more complex tasks and improved instruction-following abilities enable LLMs to better understand user intents, current LLM systems are by design incapable of capturing diverse and ever-changing personal profiles of distinct users that are implicitly encoded in life-long human-AI interaction histories. For example, an AI assistant must have rich information of a user’s residential address, personal agenda, income and consumption habits, preferences for foods and restaurants, etc., to generate a satisfactory response for a query as simple as "help me reserve a restaurant for dinner." Moreover, many aspects, including personalities, intents, preferences, etc., in user profiles are dynamic and ever-changing in the real world, making it crucial for LLM systems to be capable of constantly adjusting to the changes of user profiles. Therefore, we argue that stronger capabilities of LLMs are not all we need for AGI and life-long personalization of LLMs is another important building block for AI systems that are helpful, satisfactory, and engaging for everyone. While personalization has been carefully investigated in other domains such as recommendation systems ( Barkan et al., 2020 ; Woźniak et al., 2024 ; Dai et al., 2024 ; Li et al., 2024 ; Yang et al., 2023b ; Kang et al., 2023 ) , research on LLM personalization is quite limited and suffers from three major limitations: First, most recent work on LLM personalization ( Mysore et al., 2023 ; Zhiyuli et al., 2023 ; Yang et al., 2023a ) are task-specific and the methodology for LLM personalization can not be generalized to other tasks; Second, most existing LLM personalization methods requires training either the entire LLM or a few blocks within the LLM, making them impossible or very costly to scale to real-world applications used by million of users on a daily basis; Finally, the lack of diverse and realistic benchmarks makes it hard to evaluate new methods. To be specific, most recent work conducts experiments on the LaMP benchmark ( Salemi et al., 2024b ) . However, the tasks in this benchmark such as citation identification and scholarly title generation are very different from queries asked by real-world users and therefore not representative enough. Moreover, recent study ( Salemi et al., 2024a ) has shown that a non-personalized LLMs can also achieve competitive performance on these tasks, suggesting that the benchmark is not suitable for evaluating advanced LLM personalization approaches. In addition, all previous studies consider LLM personalization as a one-time task. However, it is crucial for LLM systems to constantly adjusting to ever-changing user profiles in real-world applications by learning from human-AI interactions. In this paper, we introduce the task of life-long personalization of LLMs and provide a detailed task formulation emphasizing the difference from previous static LLM personalization problems. We present AI Persona , a simple, general, effective, and scalable framework to build life-long personalized LLM systems or language agents. Specifically, we define user profiles as learnable dictionaries where the keys are fields representing various aspects of a real-world user, including demographics, personality, usage patterns, and preferences. The values are the users’ personal information in the corresponding fields. During inference, a user’s AI persona will be dynamically assembled into a part of the prompt/input to the LLM backbone so that it will generate personalized responses. Life-long adaptation of user profiles is achieved by a carefully designed LLM-based persona optimizer, which constantly adjust the AI Persona of the user during the interaction between the user and the LLM system. To the best of our knowledge, the proposed framework is the first method in the literature that can constantly adjust the profiles of each users during the progress of human-AI interaction. Moreover, our approach does not involve model training and only requires to store a lightweight config file for each user, making it scalable for real-world applications with large amount of users. To test the effectiveness of the proposed AI Persona framework and facilitate future research, we build PersonaBench , a benchmark for (life-long) LLM personalization research. PersonaBench consists of diverse and realistic user profiles and user queries generated with a carefully designed LLM-based workflow. Our experiments on PersonaBench demonstrates that the proposed AI Persona framework 2 2 2 We use AI Persona and Persona Learning interchangeably in this paper. can effectively learn and adapt to user profiles over time. Our contributions can be summarized as follows: 1. We provide a clear definition of life-long personalization, emphasizing the necessity of continuous adaptation in understanding user needs. To our knowledge, this is the first work to address this task in the literature. 2. We propose a novel pipeline for generating realistic persona chat data, encompassing diverse persona configurations and authentic user-agent interaction simulations. 3. We introduce a life-long personalized agent framework, serving as a baseline solution for this task. 4. We conduct experiments that demonstrate the effectiveness of our methods, showcasing significant improvements in agent performance and adaptability to user personas. By addressing these aspects, our work aims to advance the field of personalized conversational agents, fostering more effective and meaningful user interactions. 2 Related Work 2.1 Personalized LLMs The personalization of language models (LLMs) has garnered significant interest in industries such as recommendation systems and search, focusing

Read the complete text