A Language-Driven Framework for Improving Personalized Recommendations: Merging LLMs with Traditional Algorithms DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Recommender systems

Aaron Goldstein, Ayan Dutta · 2025

Read A Language-Driven Framework for Improving Personalized Recommendations: Merging LLMs with Traditional Algorithms DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Recommender systems by Aaron Goldstein, Ayan Dutta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07251.txt.

Introduction

Aaron Goldstein and Ayan Dutta let a language model act as the friend who knows your taste in movies, in a 2025 paper. Classic recommenders cannot use preferences stated in words, like a love of light-hearted comedies. Their framework starts from standard factorization methods that learn from ratings, then has a language model refine the picks using the viewer's stated or inferred taste profile.

Tested on a standard movie ratings set, the enhanced system beat the plain methods on every score the team used, with hit rates up to about six times higher in one measure. Profiles could be typed in by hand or built automatically from favorite films. The gains cost a slight rise in computing time.

Catalog record

CREATORAaron Goldstein, Ayan Dutta
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07251

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Language-Driven Framework for Improving Personalized Recommendations: Merging LLMs with Traditional Algorithms DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Recommender systems Aaron Goldstein, Ayan Dutta Abstract . Traditional recommendation algorithms are not designed to provide personalized recommendations based on user preferences provided through text, e.g., "I enjoy light-hearted comedies with a lot of humor". Large Language Models (LLMs) have emerged as one of the most promising tools for natural language processing in recent years. This research proposes a novel framework that mimics how a close friend would recommend items based on their knowledge of an individual’s tastes. We leverage LLMs to enhance movie recommendation systems by refining traditional algorithm outputs and integrating them with language-based user preference inputs. We employ Singular Value Decomposition (SVD) or SVD++ algorithms to generate initial movie recommendations, implemented using the Surprise Python library and trained on the MovieLens-Latest-Small dataset. We compare the performance of the base algorithms with our LLM-enhanced versions using leave-one-out validation hit rates and cumulative hit rates. Additionally, to compare the performance of our framework against the current state-of-the-art recommendation systems, we use rating and ranking metrics with item-based stratified 0.75 train, 0.25 test split. Our framework can generate preference profiles automatically based on users’ favorite movies or allow manual preference specification for more personalized results. Using an automated approach, our framework overwhelmingly surpassed SVD and SVD++ on every evaluation metric used (e.g., improvements of up to ∼ 6 \sim 6 x in cumulative hit rate, ∼ 3.7 \sim 3.7 x in NDCG, etc.), albeit at the cost of a slight increase in computational overhead. 1. Introduction Recommendation systems ( resnick1997recommender ) are integral parts of modern-day online experiences ranging from e-commerce to OTT platforms ( aggarwal2016introduction ; chiang2024recommender ; roy2022systematic ) . Given the recent rise in capabilities of Large Language Models (LLMs) ( achiam2023gpt ; katz2024gpt ; nori2023capabilities ) , we can utilize them to understand natural language-based user preferences before providing final recommendations to customers. Certain companies such as Amazon are already giving such options on their Fire devices showing suggestions like "Show me Al Pacino’s action movies". Despite the already exciting number of studies that focus on combining LLMs and recommendation systems (RS), extensively reviewed by ( wu2024survey ) , there is still much room for further advancement and innovation. As outlined by Wu et al. ( wu2024survey ) , LLM-based recommendation approaches can be categorized into three main paradigms: (1) LLM Embeddings + RS, where LLMs serve as feature extractors, converting features into embeddings which are then used by RS systems to improve their performance ( wu2021empowering ) ; (2) LLM Tokens + RS, where LLMs generate tokens that capture nuanced user preferences and item characteristics, this tokenized form can be used for recommendation refinement tasks as in the framework we present here ( liu2024once ) ; and (3) LLM as RS, where LLMs directly generate recommendations ( dai2023uncovering ) . Our framework primarily follows the second paradigm, using (non-tuning) LLM-generated semantic insights to improve and refine the outputs of any traditional algorithms which provides top N output recommendations. To incorporate natural language-based user preferences into the final recommendations, in this paper, we propose using an LLM-based architecture. The LLM is used in four key places: 1) in movie information retrieval through fuzzy matching when IMDb IDs are outdated or broken, 2) if movie descriptions are not available online after retrieval attempts, we use the LLM to generate concise descriptions, 3) in automatic generation of user preference profiles when users cannot or choose not to provide preferences manually, and 4) in similarity scoring of titles recommended by existing state-of-the-art SVD-based recommendations ( koren2009matrix ) . The similarity scoring system helped us to fine-tune the SVD or SVD++ ( koren2008factorization ) recommendations and provide users with better-quality recommendations. The scoring system is designed by us and provided to the LLM as an example to begin with. The LLM learned from those examples and provided recommendations based on the scoring system (see Section 2.2 ). We have implemented the proposed framework on the MovieLens-Latest dataset and compared hit rate as well as various ranking metrics such as Normalized Discounted Cumulative Gain (NDCG) and Mean Average Precision (MAP), among others. Results show that our presented LLM-enhanced recommendations comprehensively outperform the base SVM and SVD++ recommendations in all the used metrics. The main contributions of our paper can be summarized as follows. • Our novel framework graciously handles user preferences provided to it via natural language. Furthermore, we account for popularity preferences in prompting for bias mitigation. • We present a novel framework that leverages LLM to fine-tune recommendations provided by a base algorithm to the users. LLM is also used in a novel fashion to create descriptions of movies that are unavailable or in information retrieval when IMDb IDs are broken. • Results show that the proposed framework thoroughly outperforms the base algorithms in terms of hit rate and four other ranking metrics, including NDCG and MAP. 2. Methodology 2.1. Preliminaries This research proposes a framework that leverages the capabilities of Large Language Models to enhance recommendation systems by refining the recommendations of traditional recommendation algorithms such as SVD and SVD++. This approach builds on the foundational work of the studies discussed in the related works section. The proposed framework overview is illustrated in Fig. 1 . Figure 1. Framework Architecture Flowchart showing data generation and recommendation paths Data Preparation. In our hit rate and ranking metric experiments where we generate user preference profiles for all dataset users, we utilize the MovieLens Latest Small (100k dataset) ( harper2015movielens ) , which provides a comprehensive set of user ratings for recommendations. The stable 100k version from 1998 is outdated and contains old and no longer functioning IMDb links. This is why we chose the Latest Small Version (last updated 9/2018) that is more up to date with current information (i.e, more likely to contain accurate IMDb links), and contains newer and relevant movies. This dataset’s CSV files are loaded into Pandas DataFrames using Python, and we create mapping dictionaries between MovieLens Movie IDs and IMDb IDs using the links.csv included in the MovieLens dataset. This facilitates seamless integration with IMDb data, enabling the retrieval of additional movie information, including plots and reviews if needed. For our study, using Latest-Small instead of MovieLens-32M was necessary for fully comparing hit rate between LLM Enhanced and the corresponding base traditional algorithms, as generating similarity scores and user preference profiles for all users in the full MovieLens dataset would be unfeasible due to our local resource capacity. Traditional Recommendation Algorithms. We employ both SVD and SVD++ algorithms to generate initial movie recommendations. SVD and SVD++ are matrix factorization techniques that uncover latent factors in user-item interactions, making them effective for handling sparse data ( koren2009matrix ) . For calculating hit rate, the algorithms are trained on the MovieLens-Latest-Small dataset, and we compare their performance with our LLM-enhanced versions using hit rate ( AcharyaLLM2023 ) and other ranking metrics such as NDCG ( zangerle2022evaluating ) . LLM-enhanced Framework Description Generation. The integration of generating descriptions is inspired by Acharya et al. ( AcharyaLLM2023 ) . We use LLMs to generate movie descriptions through few-shot prompting when

Read the complete text