User Profile with Large Language Models: Construction, Updating, and Benchmarking
Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay · 2025
Read User Profile with Large Language Models: Construction, Updating, and Benchmarking by Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10660.txt.
Introduction
Nusrat Jahan Prottasha and co-authors build data sets and methods for user profiling with large language models, in a 2025 paper. Personalized systems need profiles that are accurate at first build and stay current as new facts arrive. The team released two open data sets, one for building profiles and one for updating them. Both aim to give researchers a shared test bed for dynamic settings.
Their method frames profiling as probability prediction from raw text. That framing lets the model weigh context and produce precise, grounded profiles. Tests with two open seven-billion-parameter models scored strongly on both tasks. Precision and recall of the generated profiles rose over baselines. The authors say language models can keep profiles fresh with less hand work. The data sets are public for further benchmarking.
Catalog record
| CREATOR | Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10660
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
User Profile with Large Language Models: Construction, Updating, and Benchmarking Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay Abstract User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach. 1 Introduction Figure 1: Overview of User Profile Management. Panel (a) shows profile construction from initial user data using an LLM, while panel (b) illustrates profile updating with new user information, maintaining dynamic User Profile Memory. User profiling is the process of constructing structured representations of individuals’ preferences, behaviors, and attributes based on available data. In this study, we define a user profile as a collection of key-value pairs encapsulating information such as demographic details, interests, behaviors, and preferences. These profiles serve as the foundation for delivering personalized recommendations and user experiences across digital platforms. In the digital era, understanding the complexities of user profiling on social media platforms is crucial. The pervasive use of user data for targeted advertising, personalized services, and social analytics has spurred the need for advanced, adaptable profiling strategies Alt et al. 2009 ; Carrara and Orsi 2011 ; Suh et al. 2005 ; Yu et al. 2012 ; Bartolomeo et al. 2008 ; Raad et al. 2010 ; Hazimeh et al. 2019 . This task encompasses diverse factors, including users’ occupations, educational backgrounds, and behavioral patterns Ochirbat et al. 2018 ; Preoţiuc-Pietro et al. 2015 ; Chicaiza et al. 2015 . Insights from these factors are vital for optimizing search algorithms Agichtein et al. 2006b , enhancing friend recommendation systems Xu et al. 2011 ; Ahmadian et al. 2019 , developing online marketing strategies Saransomrurtai 2011 ; Dennis et al. 2016 , and advancing computational social science Zhang et al. 2020 ; Brandt et al. 2020 . Recent studies emphasize the growing complexity of user profiling due to the vast and diverse nature of user-generated content on social media Luca 2015 ; Ding et al. 2016 . This complexity necessitates the integration of multiple data sources and advanced computational methods to effectively capture dynamic user behaviors Alt et al. 2009 ; Carrara and Orsi 2011 . Although regularly updating user profiles is crucial Cameron 2003 , there is a lack of open-source datasets and research focusing on efficient methods to reflect users’ evolving preferences Raad et al. 2010 . Maintaining accurate profiles in a rapidly changing digital landscape is challenging, particularly for applications like natural language processing systems that rely heavily on user data Hazimeh et al. 2019 . Social media platforms have become central to daily life, facilitating global communication and offering services from personal expression to community engagement Zhang et al. 2020 . These platforms support various activities such as networking, multimedia sharing, and content consumption, leading to an explosion of user-generated content Brandt et al. 2020 . This content provides rich data sources for user profiling research Preoţiuc-Pietro et al. 2015 . Early research treated user profiling as a multi-class classification problem, predicting characteristics like gender, age, and political orientation Al Zamal et al. 2012 ; Kowsher et al. 2021 ; Kowsher et al. 2020 . These models primarily used user-generated content and social interaction features Suh et al. 2005 . However, the field has progressed from basic demographic predictions to more nuanced analyses of user behaviors and preferences Ochirbat et al. 2018 . Recent advancements in deep learning have improved user profiling by providing deeper insights into user behavior Ding et al. 2016 . The field is moving beyond traditional approaches to explore patterns such as content consumption trends, sentiment dynamics in online interactions, and purchasing behavior changes over time Saransomrurtai 2011 . Emerging techniques like knowledge graph embeddings Wang et al. 2017 and graph neural networks Wu et al. 2020 are increasingly used to integrate external knowledge and context, further refining user profiles Godoy and Amandi 2005 . In this paper, we introduce two high-quality benchmark datasets specifically designed for user profile modeling: one for profile construction and another for profile updating. These datasets address a critical gap in current research by providing robust resources for evaluating user profiling techniques in dynamic scenarios. Building on these datasets, we propose a novel approach that utilizes LLMs for both user profile construction and updating (Figure 1 ). Our methodology employs a probabilistic framework to predict user profiles from input text, enabling precise and context-aware profile generation. This dual focus on static construction and dynamic updating reflects real-world needs, where user preferences and behaviors continuously evolve. Our experiments demonstrate that models like Mistral-7b and Llama2-7b excel in both tasks. LLMs significantly enhance the precision and recall of generated profiles, and our evaluation metrics confirm the effectiveness of this approach. These findings highlight the potential of LLMs to advance user profiling by providing more accurate, adaptable, and context-sensitive models. The primary contributions of this work are as follows: • We introduce two high-quality open-source benchmark datasets for user profile construction and updating, offering comprehensive resources for evaluating profiling techniques. • We show a methodology leveraging LLMs to address both static construction and dynamic updating of user profiles, effectively capturing evolving user information. • We conduct extensive evaluations of multiple LLMs, demonstrating their efficacy in user profiling tasks and providing insights into model performance. • We present a dynamic profile updating mechanism that maintains the accuracy and relevance of user profiles over time, with significant implications for personalized systems and social analytics. 2 Related Work User profiling is essential for personalized systems Shen et al. 2005 ; Yao et al. 2020 ; Zhu et al. 2008 , enabling platforms to deliver tailored recommendations Balog et al. 2019 ; Lu et al. 2015 ; Middleton et al. 2004 and computational social media analysis Arunachalam and Sarkar 2013 ; Bamman et al. 2014 ; Tang et al. 2015 ; Al Marouf et al. 2020 . Over time, profiling methods have evolved from rule-based systems to machine learning models, and now to LLMs, which offer improved accuracy and adaptability Bloedorn and Mani 1998 ; Wu et al. 2024 . Early research treated user profiling as a classification problem , focusing on predicting fixed attributes like gender Liu et al. 2012 ; Rao et al. 2010 ; Liu and Ruths 2013b ; Sakaki et al. 2014 ; Priadana et al. 2020 , age Rosenthal and McKeown 2011 ; Sap et al. 2014 ; Chen et al. 2015 ; Fang et al. 2015 ; Mac Kim et al. 2017 ; Bessarab et al. 2025 , and political views Rao et al. 2010 ;