Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation

Nanshan Jia, Chenfei Yuan, Yuhang Wu, Zeyu Zheng · 2025

Read Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation by Nanshan Jia, Chenfei Yuan, Yuhang Wu, Zeyu Zheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11658.txt.

Introduction

Nanshan Jia and three co-authors use language models to sharpen product suggestions, in a 2025 arXiv paper. Many shops already run a base tool that learns from click history. Bolting a big language model straight on raises doubts about why it picks what it picks. The team instead uses the model as a side helper that mimics sales logic in plain terms.

The helper reads item traits that people can grasp and builds a guided code for each item. That code merges with a slimmed form of the base tool's code, and the mix feeds the final picks. Tests across tasks show gains of roughly ten to fifty percent on standard scores. Case studies, the team adds, show the picks are easier to explain too.

Catalog record

CREATORNanshan Jia, Chenfei Yuan, Yuhang Wu, Zeyu Zheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-11658

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation Nanshan Jia, Chenfei Yuan, Yuhang Wu, Zeyu Zheng Abstract The fast development of Large Language Models (LLMs) offers growing opportunities to further improve sequential recommendation systems. Yet for some practitioners, integrating LLMs to their existing base recommendation systems raises questions about model interpretability, transparency and related safety. To partly alleviate challenges from these questions, we propose guided embedding refinement, a method that carries out a guided and interpretable usage of LLM to enhance the embeddings associated with the base recommendation system. Instead of directly using LLMs as the backbone of sequential recommendation systems, we utilize them as auxiliary tools to emulate the sales logic of recommendation and generate guided embeddings that capture domain-relevant semantic information on interpretable attributes. Benefiting from the strong generalization capabilities of the guided embedding, we construct refined embedding by using the guided embedding and reduced-dimension version of the base embedding. We then integrate the refined embedding into the recommendation module for training and inference. A range of numerical experiments demonstrate that guided embedding is adaptable to various given existing base embedding models, and generalizes well across different recommendation tasks. The numerical results show that the refined embedding not only improves recommendation performance, achieving approximately 10 % 10\% to 50 % 50\% gains in Mean Reciprocal Rank (MRR), Recall rate, and Normalized Discounted Cumulative Gain (NDCG), but also enhances interpretability, as evidenced by case studies. 1 Introduction Our work considers recommendation systems in the context of e-commerce, where good recommendations have been essential for improving customer shopping experiences Resnick & Varian (1997) ; Schafer et al. (1999) ; Jannach (2010) . In a recommendation system, items refer to the entities being recommended and users represent the main recipients of those recommendations. Interactions denote the relationships between users and items, such as viewing, purchasing, rating, or commenting. A common recommendation task involves assessing whether a specific item should be recommended to a user, based on the item’s characteristics and the user’s past interaction history. Among various types of recommendation systems, sequential recommendation has taken an increasingly important role in recent years. One characteristic of sequential recommendation is that a user’s past interaction history can be represented in chronological order. The chronological order allows the recommendation system to flexibly assign different weights to each interaction based on its relative position in the interaction history, allowing for a more accurate analysis and a better capture of user preferences over time Ezeife & Karlapalepu (2023) . Due to the assistance of Large Language Models (LLMs) and transformer architecture, the performance of sequential recommendation systems has been improved in two key ways. First, LLMs provide improvement for the embedding of users and items. Traditional recommendation systems rely on ID-based embeddings, where users and items are represented as unique IDs in a shared latent space Goldberg et al. (1992) ; Sarwar et al. (2001) ; Koren et al. (2009) . Although these embeddings are effective for calculating similarity and making recommendations, they have not adequately utilized the language-based information, especially so when a new user or item with limited interaction history enters the recommendation system Sarwar et al. (2001) ; Kim et al. (2024) . LLMs can utilize their power to improve the understanding of contextual information about users and items. These information are prepared as prompts and converted into embeddings, which can then be used for subsequent recommendation tasks. Second, the transformer architecture, along with its attention mechanism, has proven to be effective for sequential tasks Vaswani (2017) . By employing transformers as the backbone model for sequential recommendation, these recommendation systems achieve superior performance compared to previous methods based on Markov chains and recurrent neural networks (RNNs) Kang & McAuley (2018) ; Sun et al. (2019) . Although the integration of LLMs has improved the performance of sequential recommendation systems, the black-box nature of LLMs raises questions about model complexity and the transparency of decision-making process. Because of this, some practitioners feel not as certain about using LLM-based sequential recommendation and they have expressed a desire for improved interpretability and explainability in the use of LLM for recommendation tasks Kuosmanen (2020) . Several studies Gao et al. (2023) ; Lei et al. (2023) ; Luo et al. (2024) ; Shi et al. (2024) focus on improving the explainability of LLMs in the context of recommendation systems. These approaches either leverage prompt engineering to generate explanations directly from the API Gao et al. (2023) or fine-tune pre-trained LLMs for explanation Lei et al. (2023) . Both approaches offer contextual explanations for recommendations, thereby enhancing model explainability. They share the advantage of providing direct, word-based explanations, making them straightforward and easy to understand for users of the recommendation system. Despite the aforementioned benefits, the existing works on improving explainability of LLM-assisted recommendations have certain challenges, which may be summarized as follows. First, these models generate explanations by producing sentences with detailed semantic content. The text generation process introduces extra computational overhead, increasing the overall inference time. When the generated explanations are relatively long, this added complexity can lead to delays in generating recommendations, which may not be appropriate for online applications or scenarios where real-time or near-instantaneous recommendations are critical. Second, some explainable recommendation models do not lead to a direct improvement in the overall recommendation performance. For example, Gao et al. (2023) uses the OpenAI API to generate explanations, but this process is separate from the recommendation system and does not improve the quality of the recommendations. Third, many existing explainable recommendation models are difficult to generalize across different recommendation tasks. These models are often trained or fine-tuned for a specific task on a particular dataset. Although they can generate high-quality explanations within that context, they are not typically designed to generate explanations for other types of recommendation tasks. This limits their versatility, as applying them to new tasks requires retraining or fine-tuning multiple pre-trained LLMs. In this work, we propose an embedding refinement method designed to enhance the interpretability of recommendations while preserving key advantages such as fast inference speed, improved recommendation performance, and adaptability across various recommendation tasks. Before summarizing our method, we would like to briefly discuss some of the thoughts that motivated this work. Many recommendations in the era without internet were made by experienced sales persons. For recommending clothes, an experienced sales person would evaluate a customer’s purchasing attributes, such as spending budget, willingness for luxury items, preference for durability versus trendiness, brand loyalty, etc. For recommending other categories of goods, each category may have accumulated in the past some particular purchasing attributes that are useful to better assess the customer’s preferences. Our work tries to explicitly use such purchasing attributes to guide the use of LLMs to generate embeddings for customers and for items. Our work takes a base recommendation system as given, where such base recommendation system is capable of embedding both users and items, and generating recommendations based on those embeddings; for example, SASRec Kang & McAuley (2018) , Bert4Rec Sun et al. (2019) and GRU4Rec Hidasi (2015) . The embeddings produced by the base recommendation system are referred to as base embeddings. To enhance the interpretability and performance of the base

Read the complete text