Enhancing ID-based Recommendation with Large Language Models DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Information systems Language models CCS: Information systems Recommender systems
Lei Chen, Chen Gao, Xiaoyi Du, Hengliang Luo, Depeng Jin, Yong Li, Meng Wang · 2024
Read Enhancing ID-based Recommendation with Large Language Models DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Information systems Language models CCS: Information systems Recommender systems by Lei Chen, Chen Gao, Xiaoyi Du, Hengliang Luo, Depeng Jin, Yong Li, Meng Wang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02041.txt.
Description
Abstract . Large Language Models (LLMs) have recently garnered significant attention in various domains, including recommendation systems. Recent research leverages the capabilities of LLMs to improve the performance and user modeling aspects of recommender systems. These studies primarily focus on utilizing LLMs to interpret textual data in recommendation tasks.
Catalog record
| CREATOR | Lei Chen, Chen Gao, Xiaoyi Du, Hengliang Luo, Depeng Jin, Yong Li, Meng Wang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02041
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing ID-based Recommendation with Large Language Models DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Information systems Language models CCS: Information systems Recommender systems Lei Chen, Chen Gao, Xiaoyi Du, Hengliang Luo, Depeng Jin, Yong Li, Meng Wang Abstract . Large Language Models (LLMs) have recently garnered significant attention in various domains, including recommendation systems. Recent research leverages the capabilities of LLMs to improve the performance and user modeling aspects of recommender systems. These studies primarily focus on utilizing LLMs to interpret textual data in recommendation tasks. However, it’s worth noting that in ID-based recommendations, textual data is absent, and only ID data is available. The untapped potential of LLMs for ID data within the ID-based recommendation paradigm remains relatively unexplored. To this end, we introduce a pioneering approach called "LLM for ID-based Recommendation" (LLM4IDRec). This innovative approach integrates the capabilities of LLMs while exclusively relying on ID data, thus diverging from the previous reliance on textual data. The basic idea of LLM4IDRec is that by employing LLM to augment ID data, if augmented ID data can improve recommendation performance, it demonstrates the ability of LLM to interpret ID data effectively, exploring an innovative way for the integration of LLM in ID-based recommendation. Specifically, we first define a prompt template to enhance LLM’s ability to comprehend ID data and the ID-based recommendation task. Next, during the process of generating training data using this prompt template, we develop two efficient methods to capture both the local and global structure of ID data. We feed this generated training data into the LLM and employ LoRA for fine-tuning LLM. Following the fine-tuning phase, we utilize the fine-tuned LLM to generate ID data that aligns with users’ preferences. We design two filtering strategies to eliminate invalid generated data. Thirdly, we can merge the original ID data with the generated ID data, creating augmented data. Finally, we input this augmented data into the existing ID-based recommendation models without any modifications to the recommendation model itself. We evaluate the effectiveness of our LLM4IDRec approach using three widely-used datasets. Our results demonstrate a notable improvement in recommendation performance, with our approach consistently outperforming existing methods in ID-based recommendation by solely augmenting input data. 1. Introduction Recommender systems play a central role and have emerged as indispensable tools in online services ( Covington et al. 2016 ; Guo et al. 2017 ; Dai et al. 2021 ; Yu et al. 2023b ; Chen et al. 2023 ) . They serve the critical function of offering personalized recommendations in the face of information overload, effectively aligning with user preferences across a range of tasks ( Zhou et al. 2018 ; Chen et al. 2020b ) . While various recommendation tasks exist ( Covington et al. 2016 ; Lin et al. 2021 ; Xi et al. 2023 ) , including top-N recommendation, next-item recommendation, and rating prediction, the common approach involves learning user representations to model their preferences and intentions. These learned representations are subsequently employed to generate decisions regarding recommended items for users. In recent years, Large Language Models (LLMs) have exhibited remarkable proficiency in approximating human intentions and excelling in a wide array of tasks, including reasoning and decision-making ( Huang and Chang 2022 ; Brown et al. 2020b ; Zhao et al. 2023 ) . Inspired by the great success of Large Language Models (LLMs) ( Brown et al. 2020a ; Touvron et al. 2023 ; Chiang et al. 2023 ) , exploring the potential of LLMs in recommendation is attracting attention ( Li et al. 2023d ; Bao et al. 2023 ; Geng et al. 2022a ; Cui et al. 2022 ; Dai et al. 2023a ; Zhang et al. 2024a ; Zhao et al. 2024 ) , especially driven by innate reasoning capabilities and approximating human intentions of LLMs. With the exploration of LLM-based recommendation ( Gao et al. 2023 ; Hou et al. 2023 ; Liu et al. 2023b ) , this direction has emerged as a promising approach for the next-generation recommendation systems ( Zeng et al. 2021 ; Liu et al. 2023c ; Wang et al. 2023 ) . Figure 1. An example of comparing different structures of LLM in Recommender Systems (RS). Figure 1(a) and (b) depict current approaches for incorporating LLM into recommendation models. Both methods utilize textual data as input, such as user and item profiles. Nevertheless, in the context of ID-based recommendations, solely ID data is available, devoid of any textual information. When dealing with recommendation scenarios that solely rely on ID data, we investigated the application of LLM for ID-based recommendation, as shown in Figure 1(c). In the context of recommender systems, accurately capturing patterns in user interaction data (ID data) is crucial for generating relevant recommendations. The core goal of recommendation algorithms is to predict the items a user is likely to engage with based on their interaction history. Over the years, various methods have been developed to improve recommendation performance, including Markov Chains ( He and McAuley 2016 ; Rendle et al. 2010 ) , RNN/CNN models ( He and McAuley 2016 ; Rendle et al. 2010 ) , self-attentive models ( Kang and McAuley 2018 ; Li et al. 2020 ; Zhao et al. 2021 ) , and more recently, graph convolutional network (GCN)-based models ( Wang et al. 2019 ; Chen et al. 2020a ; Yu et al. 2022 ) . These advancements stem from increasingly sophisticated architectures designed to better capture the complex relationships in interaction data. GCN-based models, for example, excel at capturing intricate relationships between users and items compared to traditional CNN-based models. However, the emergence of LLMs introduces a new paradigm with even greater capacity for modeling and reasoning, surpassing existing structures like GCNs ( Minaee et al. 2024 ; Wang et al. 2024 ; Ren et al. 2024 ; Wu et al. 2024 ) . Given LLMs’ ability to model and interpret complex data, they present a promising direction for processing user interaction data (ID data). Various studies have demonstrated that LLMs can successfully address graph-related tasks, which rely heavily on understanding and reasoning about relationships within ID data ( Zhang et al. 2024b ; Guo et al. 2023 ; Ye et al. 2024 ; Geng et al. 2022b ; Sun et al. 2024 ) . For instance, LLM4DyG ( Zhang et al. 2024b ) constructs prompts using node and edge IDs, which are then processed by LLMs for inference, effectively tackling graph-based tasks. This suggests that employing LLMs to model and interpret ID data is not only feasible but also a research direction worth exploring further. For the studies of LLMs in recommendations, the current approaches can be broadly categorized into two groups. As shown in Figure 1 (a), the first category involves leveraging LLMs as enhanced feature extractors to improve the performance of recommendation models in terms of accuracy or interpretability ( Li et al. 2023c ; Li et al. 2023b ; Liu et al. 2023a ; Wei et al. 2023 ) . For example, LLMRec ( Wei et al. 2023 ) uses LLMs to explicitly model with three types of features, such as user profile. In this category, LLMs serve as tools to