ExplainRec: Towards Explainable Multi-Modal Zero-Shot Recommendation with Preference Attribution and Large Language Models

1 st Bo Ma*, 2 nd Hang Li, 3 rd ZeHua Hu, 4 th XiaoFan Gui, 5 th LuYao Liu, 6 th Simon Lau · 2025

Read ExplainRec: Towards Explainable Multi-Modal Zero-Shot Recommendation with Preference Attribution and Large Language Models by 1 st Bo Ma*, 2 nd Hang Li, 3 rd ZeHua Hu, 4 th XiaoFan Gui, 5 th LuYao Liu, 6 th Simon Lau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14770.txt.

Introduction

Bo Ma, LuYao Liu, ZeHua Hu, and Simon Lau present ExplainRec, a recommendation framework that explains its suggestions and works for brand-new users and items. Language model recommenders have opened fresh possibilities, but leading approaches struggle with explainability and with the cold-start problem. ExplainRec extends them with four components: tuning that attributes a recommendation to the user's preferences, zero-shot transfer of those preferences to new cases. Multi-modal fusion of visual and textual content, and multi-task training that optimises the parts together.

On a large movie data set and an Amazon data set. The authors report gains over existing methods in ranking accuracy for both movie and cross-domain tasks, while also producing interpretable explanations and handling cold starts. The combination targets the trust problem: a recommendation a user can interrogate is easier to accept.

Catalog record

CREATOR1 st Bo Ma*, 2 nd Hang Li, 3 rd ZeHua Hu, 4 th XiaoFan Gui, 5 th LuYao Liu, 6 th Simon Lau
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14770

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ExplainRec: Towards Explainable Multi-Modal Zero-Shot Recommendation with Preference Attribution and Large Language Models 1 st Bo Ma*, 2 nd Hang Li, 3 rd ZeHua Hu, 4 th XiaoFan Gui, 5 th LuYao Liu, 6 th Simon Lau Abstract Recent advances in Large Language Models (LLMs) have opened new possibilities for recommendation systems, though current approaches such as TALLRec face challenges in explainability and cold-start scenarios. We present ExplainRec, a framework that extends LLM-based recommendation capabilities through preference attribution, multi-modal fusion, and zero-shot transfer learning. The framework incorporates four technical contributions: preference attribution tuning for explainable recommendations, zero-shot preference transfer for cold-start users and items, multi-modal enhancement leveraging visual and textual content, and multi-task collaborative optimization. Experimental evaluation on MovieLens-25M and Amazon datasets shows that ExplainRec outperforms existing methods, achieving AUC improvements of 0.7% on movie recommendation and 0.9% on cross-domain tasks, while generating interpretable explanations and handling cold-start scenarios effectively. I Introduction Large Language Models (LLMs) have gained considerable attention in recommendation systems research due to their semantic understanding and generalization capabilities [ 1 , 2 ] . Works such as TALLRec [ 1 ] have shown that LLMs can be adapted for recommendation tasks through instruction tuning, achieving performance comparable to traditional collaborative filtering approaches. Despite these advances, current LLM-based recommendation methods exhibit several limitations that limit their practical applicability: Limited Explainability: Methods such as TALLRec generate binary predictions without providing reasoning for their recommendations. This lack of transparency poses challenges for applications requiring interpretable results, especially in domains where user trust and regulatory requirements are important [ 3 , 4 ] . Cold-Start Problem: Current approaches face difficulties when dealing with new users or items having limited interaction history. Although TALLRec can operate with few-shot learning using 16 samples, it cannot handle zero-shot scenarios where historical data is unavailable [ 5 , 6 ] . Single-Modal Limitations: Most existing LLM-based recommenders process only textual information, overlooking multi-modal content including images, videos, and audio that characterize contemporary recommendation applications [ 7 , 8 ] . Task Isolation: Current frameworks train separate models for different recommendation tasks (rating prediction, ranking, explanation generation), leading to inefficient resource utilization and missed opportunities for knowledge sharing across related tasks [ 9 , 10 ] . To address these challenges, we develop ExplainRec , a framework that extends LLM-based recommendation systems through four technical contributions: 1. Preference Attribution Tuning: An extension of instruction tuning that incorporates preference reasoning, allowing models to predict user preferences while providing explanations based on item attributes and user history. 2. Zero-Shot Preference Transfer: A universal preference knowledge base combined with meta-learning mechanisms to handle recommendations for new users or items without historical interactions. 3. Multi-Modal Enhancement: Integration of visual and textual modalities through unified embedding spaces to leverage multi-modal content for improved recommendation accuracy. 4. Multi-Task Collaborative Optimization: A joint training framework that optimizes multiple recommendation tasks simultaneously, facilitating knowledge sharing and improving system efficiency. The main contributions of this work include: • Identification and systematic treatment of four limitations in existing LLM-based recommendation systems, with both theoretical analysis and practical solutions. • Development of ExplainRec, a unified framework combining explainable recommendation, zero-shot transfer, multi-modal fusion, and multi-task learning within a single LLM-based architecture. • Introduction of novel techniques including preference attribution instruction tuning, universal preference transfer mechanisms, and adaptive multi-modal fusion strategies. • Comprehensive experimental evaluation on large-scale datasets (MovieLens-25M, Amazon Product Data) showing improvements over existing methods while maintaining computational efficiency. • Analysis of explainability quality, cold-start performance, and cross-domain generalization capabilities, providing insights for future research directions. II Related Work II-A LLM-based Recommendation Systems Large Language Models have attracted growing interest for recommendation system applications. TALLRec [ 1 ] introduced instruction tuning for adapting LLMs to recommendation tasks, achieving performance competitive with traditional methods. Chat-REC [ 2 ] investigated conversational recommendation through ChatGPT, while P5 [ 11 ] developed a unified text-to-text paradigm for multiple recommendation tasks. These approaches focus primarily on textual information and provide limited explainability mechanisms. II-B Explainable Recommendation Explainable recommendation has gained increasing attention due to the need for transparency and user trust [ 3 ] . Traditional approaches include matrix factorization with explanations [ 12 ] , attention-based methods [ 13 ] , and knowledge graph-enhanced explanations [ 14 ] . Recent work has explored using LLMs for generating natural language explanations [ 15 ] , but most methods generate explanations post-hoc rather than integrating explainability into the recommendation process. II-C Multi-modal Recommendation Multi-modal recommendation systems leverage various data modalities to improve recommendation accuracy. MMGCN [ 7 ] incorporates visual and textual features using graph convolutional networks, while LATTICE [ 8 ] employs attention mechanisms for multi-modal fusion. Recent work has explored vision-language models for recommendation [ 16 ] , but integration with LLMs remains underexplored. II-D Cold-start Recommendation Cold-start recommendation addresses the challenge of recommending items to new users or recommending new items. Traditional approaches include content-based methods [ 17 ] , demographic-based filtering [ 18 ] , and transfer learning techniques [ 19 ] . Recent work has explored meta-learning for few-shot recommendation [ 20 ] and cross-domain transfer [ 21 ] , but zero-shot scenarios remain challenging. III ExplainRec Framework III-A Overview ExplainRec extends the TALLRec framework with four key enhancements designed to address the limitations of existing LLM-based recommendation systems. Our framework maintains the two-stage training paradigm (Alpaca tuning followed by recommendation tuning) while introducing novel components for explainability, zero-shot transfer, multi-modal fusion, and multi-task optimization. The overall architecture is illustrated in Fig. 1 . The overall architecture consists of: (1) a multi-modal encoder that processes textual and visual information, (2) a preference attribution module that generates explanations, (3) a zero-shot transfer component that handles cold-start scenarios, and (4) a multi-task coordinator that jointly optimizes multiple objectives. INPUT User History Item Content Target Item INNOVATION Multi-modal Enhancement Preference Attribution Zero-shot Transfer Multi-task Optimization LLM LLaMA-7B with LoRA Adaptation TRAINING Alpaca Tuning Rec-tuning Multi-task Fine-tuning OUTPUT Recommendation + Explanation Fig. 1: Overview of the ExplainRec framework. The framework extends TALLRec with four key components: (1) Multi-modal enhancement for processing text and visual information, (2) Preference attribution module for generating explanations, (3) Zero-shot transfer component for cold-start scenarios, and (4) Multi-task optimization for joint training. The framework maintains the two-stage training paradigm while adding explainability and multi-modal capabilities. III-B Preference Attribution Tuning Conventional recommendation systems generate predictions without providing explanations. Our approach extends instruction tuning to incorporate preference reasoning, allowing the model to explain the rationale behind user preferences for specific items. III-B 1 Attribution-Enhanced Instruction Format We modify the standard TALLRec instruction format to include preference reasons: \mathcal{I}_{attr}=\{I_{task},I_{input}^{attr},O_{pred},O_{reason}\} (1) where I_{task} is the task instruction, I_{input}^{attr} includes attributed user history, O_{pred} is the binary prediction, and O_{reason} is the explanation. The attributed input format becomes: I_{input}^{attr}=\{H_{liked}^{attr},H_{disliked}^{attr},T_{item}\} (2) where H_{liked}^{attr}=\{(i_{1},r_{1}),...,(i_{k},r_{k})\} contains liked items with reasons r_{j} , and similarly for disliked items. III-B 2 Preference Reasoning Loss We introduce a dual-objective loss function that jointly optimizes prediction accuracy and explanation quality: \mathcal{L}_{attr}=\alpha\mathcal{L}_{pred}+\beta\mathcal{L}_{reason}+\gamma\mathcal{L}_{consistency} (3) where \mathcal{L}_{pred} is the standard binary classification loss, \mathcal{L}_{reason} measures explanation quality using BLEU score against ground truth, and \mathcal{L}_{consistency} ensures logical consistency between predictions and explanations. III-C Zero-Shot Preference Transfer To address cold-start scenarios, we

Read the complete text