ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning

Jiani Huang, Shijie Wang, Liangbo Ning, Wenqi Fan, Qing Li · 2026

Read ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning by Jiani Huang, Shijie Wang, Liangbo Ning, Wenqi Fan, Qing Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-07851.txt.

Introduction

Huang and four colleagues build ReRec, a recommendation helper that reasons before it recommends. Language-model recommenders show promise but stumble on multi-step reasoning in hard queries. Their reinforcement fine-tuning method sharpens reasoning with dual-graph reward shaping, which blends ranking quality with query and preference alignment scores. A second piece splits outputs into reasoning segments and penalizes wrong steps. Together the pieces aim for tailored picks that show their working rather than merely listing items.

Catalog record

CREATORJiani Huang, Shijie Wang, Liangbo Ning, Wenqi Fan, Qing Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52390
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.07851

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning Jiani Huang, Shijie Wang, Liangbo Ning, Wenqi Fan, Qing Li Abstract With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec , a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping , integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation , which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler , dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec . 1 1 footnotetext: Corresponding authors. 1 Introduction With the rapid advancement of AI technologies, users now expect more intelligent, context-aware recommendation systems (RecSys) that understand complex, real-time needs and provide personalized suggestions with clear reasoning Huang et al. 2025c ; Zhang et al. 2024 . Traditional methods, such as matrix factorization (MF) and graph neural networks (GNNs) Fan et al. 2019 ; Fan et al. 2020 , rely on historical data like user ratings or clicks Chen et al. 2015 ; Fan et al. 2022 , but struggle to process natural language queries that reflect current preferences. As a result, they fail to meet the demand for intelligent recommendation assistants. Figure 1: Example of Reasoning-Augmented LLM-based Recommendation Assistant. The advent of large language models (LLMs) has unlocked new possibilities for intelligent, interactive recommendation assistants Zhao et al. 2024 ; Wang et al. 2025c . With their advanced language comprehension, generation abilities, and broad general knowledge Minaee et al. 2024 , LLMs have the potential to understand natural language user queries and generate personalized recommendations. Recent work has demonstrated this potential in developing conversational recommendation systems (CRS) that engage users in multi-turn dialogues before suggesting items Yang et al. 2024 ; Liang et al. 2024 ; Zhu et al. 2025a . However, these dialogues often involve simple and direct user queries Huang et al. 2025a , such as "Recommend me a sci-fi movie," which require minimal reasoning or constraints. In contrast, user queries often involve more complex queries that demand deeper reasoning for effective decision-making Ren et al. 2024 ; Wang et al. 2025b . Consider the complex query illustrated in Figure 1 , the assistant must first infer the correct movie from the user’s description, then identify the lead actor, and finally suggest other films featuring that actor. This process requires the model to engage in multi-step reasoning, going beyond basic attribute matching or shallow semantic understanding. Existing LLM-based RecSys are limited in handling such reasoning-intensive queries due to their inadequate capacity for deep, multi-step reasoning Shi et al. 2024 ; Tsai et al. 2024 . As a result, there is an urgent need for reasoning-augmented LLMs capable of addressing these complex user requests. Recent advances in reinforcement fine-tuning (RFT) with rule-based rewards have significantly improved LLMs’ reasoning and generalization for various tasks Xie et al. 2025 ; Ke et al. 2025 ; Zou et al. 2025 . Unlike supervised fine-tuning (SFT), which requires large amounts of labeled data, RFT uses reinforcement learning (RL) to optimize the model through self-exploration. In this process, the LLM generates responses, while a reward model evaluates them, guiding the model to reinforce effective reasoning strategies. RFT offers better generalization and reduces catastrophic forgetting compared to SFT Chu et al. 2025 , as it focuses on active reasoning rather than memorization. However, despite its potential, directly applying RFT to train reasoning-augmented LLM-based recommendation assistants for complex queries presents several challenges. One key challenge lies in developing fine-grained reward models for complex, query-based recommendation tasks . In general, the reward model in the RFT framework provides feedback on the recommendation quality, directly guiding the policy model updates. Existing studies often rely on task-specific metrics, such as NDCG, as reward signals, which can be overly stringent and sparse. For instance, when the LLM-based policy model generates recommendations that align with the user’s query but deviate from the ground truth, it receives the same zero reward, as responses that entirely fail to address the query. Such coarse rewards may potentially reduce the LLM-based policy model’s exploration efficiency, ultimately undermining its overall performance. Another challenge is the lack of supervision for the reasoning process behind the recommendations . Recent RFT methods such as GRPO Shao et al. 2024 typically assign a single reward score to the entire response. As all tokens share this unified score, the LLM policy cannot distinguish which specific parts of the reasoning were correct or flawed. This lack of supervision over intermediate reasoning steps makes it difficult for the model to identify and correct errors in its reasoning, such as misinterpreting user needs or overlooking key constraints of expected items. Consequently, the model struggles to improve its reasoning and may generate suboptimal recommendations Yang et al. 2024 ; Zhu et al. 2025a . To address these challenges, we propose a novel RFT-based framework ( ReRec ) for training a reasoning-augmented LLM-based recommendation assistant. In order to deliver fine-grained reward signals , we introduce a Dual-Graph Enhanced Reward Shaping mechanism, which enriches traditional metrics like NDCG with two additional components: the Query Alignment Score (QAS) and the Preference Alignment Score (PAS). QAS evaluates how well the recommendations satisfy explicit query constraints using an item-attribute graph, while PAS assesses alignment with user preferences based on similarity to target items. For better supervision of the reasoning process , we design Reasoning-aware Advantage Estimation , which decomposes the recommendation into reasoning steps and penalizes incorrect ones with lower advantages. Additionally, to mitigate the instability often associated with RL, we introduce the Online Curriculum Scheduler , which dynamically reorders training data by prioritizing easier queries based on previous epoch performance, ensuring smoother convergence. In summary, our contributions are: • We bridge the gap between recommendation and reasoning, enabling reasoning-augmented LLM-based recommendation assistants to understand users’ complex queries and provide reasonable, context-aware recommendations. • We propose a reinforcement fine-tuning framework ReRec that better adapts LLMs to recommendation tasks. It aligns RL signals with recommendation goals, improves reasoning feedback, and enhances training stability, enabling more accurate and context-aware recommendations. • Extensive experiments demonstrate that our method outperforms state-of-the-art baselines. Additionally, it retains strong instruction-following and reasoning capabilities, ensuring versatility for intelligent recommendations. 2 Preliminaries Problem Statement. Users often express preferences through complex, multifaceted natural language queries, which require the recommendation assistant to perform multi-step reasoning beyond simple keyword matching or attribute filtering to understand the user’s intent. Formally, given a user’s query q , the LLM-based recommendation assistant \pi_{\theta} generates a response o , which includes both the reasoning process and a recommended item p . The reasoning should explain why p was selected and why other items were excluded, defined as \pi_{\theta}(q)=o .

Read the complete text