ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation

Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Frank Shyu, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu · 2026

Read ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation by Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Frank Shyu, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28357.txt.

Introduction

Yihua Zhang and thirteen colleagues built ReasonRec, a recommender that thinks out loud. Multimodal recommenders fuse images and text well but decide opaquely and waste computation on easy cases. ReasonRec wraps recommendation in an explicit three-stage pipeline: observe the user and items, deliberate in writing, then act.

Zhang's team turns varied recommendation tasks into chain-of-thought prompts so the model states its intermediate steps, and trains it with a curriculum that grows reasoning difficulty to handle new and long-tail users. When the agent's own uncertainty runs high, it hands the query to a small classical model instead. Across four task types and five datasets, ranking quality rose by over thirty percent relative to strong rivals, while delegating about a third of queries cut latency sharply.

Catalog record

CREATORYihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Frank Shyu, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28357

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Frank Shyu, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu Abstract Recent advances in multimodal recommenders excel at feature fusion but remain opaque and inefficient decision-makers, lacking explicit reasoning and self-awareness of uncertainty. To address this, we introduce ReasonRec , a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline: Observe , via a pretrained Vision-Language Model (VLM) encoder; Deliberate , by formulating recommendation as chain-of-thought (CoT) reasoning tasks and explicitly quantifying prediction uncertainty through an evidence-horizon-aware curriculum; and Act , through dynamic delegation of uncertain or challenging queries to lightweight classical recommendation models. Specifically, we propose a reasoning-aware visual instruction tuning strategy that systematically transforms diverse recommendation tasks into unified CoT prompts, enabling the VLM to explicitly articulate intermediate decision steps. Additionally, our evidence-horizon curriculum progressively enhances the reasoning complexity to better handle cold-start and long-tail user scenarios, significantly boosting model generalization. Furthermore, the uncertainty-guided delegation mechanism empowers the agent to assess its own confidence, strategically allocating computational resources to optimize both recommendation accuracy and inference efficiency. Comprehensive experiments on four standard recommendation tasks (sequential recommendation, direct recommendation, CTR prediction, and explanation generation) across five real-world datasets demonstrate that ReasonRec achieves over 30% relative improvement in key ranking metrics ( e.g. , HR@5, NDCG@5) compared to state-of-the-art multimodal recommenders. Crucially, ReasonRec substantially reduces inference latency by dynamically delegating up to 35% of queries to efficient sub-models without compromising accuracy. Extensive ablation studies further confirm that each proposed reasoning and planning mechanism individually contributes substantially to ReasonRec ’s overall effectiveness. Collectively, our results illustrate a clear pathway towards interpretable, adaptive, and efficient multimodal recommendation through explicit reasoning and agentic design. † † affiliation: Meta AI † † affiliation: Michigan State University † † affiliation: The University of North Carolina at Chapel Hill † † contribution: Project Tead † † date: August 10, 2026 † † correspondence: [email protected] † † Accepted to: The 64th Annual Meeting of the Association for Computational Linguistics 1 Introduction Recent advances in large language models (LLMs), exemplified by DeepSeek-R1 Guo et al. 2025 , GPT-4o Hurst et al. 2024 , and Gemini Team et al. 2023a , have sparked a widespread “reasoning renaissance” in artificial intelligence research. These models leverage explicit chain-of-thought (CoT) Wei et al. 2022 reasoning to achieve remarkable performance across various complex reasoning tasks, including logical inference Li et al. 2024 , mathematical problem-solving MAA 2024 , and planning Xie et al. 2024a . Such reasoning capabilities not only improve model interpretability but also significantly enhance generalization and trustworthiness. Despite these successes, the power of explicit reasoning Guo et al. 2025 has yet to be systematically explored and exploited in multimodal recommendation Xie et al. 2024b , a crucial domain inherently requiring nuanced and transparent decision-making. Figure 1 : Performance comparison across four basic recommendation tasks and three challenging scenarios. Each task (color-coded region) is evaluated on two metrics. All results are normalized such that the best baseline performance is set to \mathbf{1.0} for each metric-task pair. Markers representing ReasonRec indicate the relative improvement ratio over the strongest baseline, with values greater than demonstrating superiority. Experiment setups and results can be found in Sec. 4 . Multimodal recommendation Cheng et al. 2023 ; Geng et al. 2023 fundamentally involves intricate multi-step deliberation processes. To effectively fulfill user needs, a recommender must first accurately infer user intent from sparse and ambiguous interactions Covington et al. 2016 ; Cheng et al. 2016 ; Wang et al. 2021a . Next, it must carefully evaluate the semantic content embedded in diverse multimodal signals, such as visual imagery and descriptive text, to estimate relevance. Finally, recommenders must explicitly reason about complex utility trade-offs, balancing user satisfaction, content freshness, and system efficiency Kang and McAuley 2018 ; Zhou et al. 2020 ; Fan et al. 2021 ; Yuan et al. 2019 . However, existing multimodal recommendation systems typically rely on a singular, black-box forward inference, collapsing all these intricate decisions into an opaque scoring mechanism Cheng et al. 2023 ; Geng et al. 2023 . This implicit approach severely limits transparency, interpretability, and robustness, especially in challenging scenarios like cold-start and cross-domain tasks Zhang et al. 2024 ; Cheng et al. 2020 ; Song et al. 2019 ; Naumov et al. 2019 . Most contemporary Vision-Language Models (VLMs) Zong et al. 2024 ; Liu et al. 2023b ; Liu et al. 2024b ; Zhu et al. 2023 ; Ye et al. 2023 ; Wang et al. 2023 ; Li et al. 2023a ; Alayrac et al. 2022 ; Awadalla et al. 2023 in recommendation embed multimodal signals implicitly within their transformer architectures. While effective at feature fusion, these models fail to explicitly articulate their reasoning processes Zhang et al. 2024 , resulting in recommendations that are hard to interpret and audit. More critically, these single-pass architectures lack the self-awareness to diagnose the uncertainty and thus cannot dynamically adapt by invoking specialized external knowledge or computationally efficient sub-models. Thus, existing multimodal recommenders suffer from poor reliability in low-confidence situations and waste computational resources by uniformly treating all recommendation decisions as equally complex. Motivated by these limitations, we propose to fundamentally rethink the design of multimodal recommenders around an explicit reasoning paradigm. Specifically, we ask: Can we build an agentic VLM that reasons explicitly through chain-of-thought, transparently assesses its own uncertainty, and dynamically delegates uncertain or challenging cases to external, lightweight specialists? Addressing this question not only promises to significantly enhance the interpretability and generalization capabilities of multimodal recommenders but also introduces a novel, reasoning-driven agent framework into the broader recommendation landscape. In this work, we introduce ReasonRec, a reasoning-augmented multimodal recommendation agent that transforms recommendation into a transparent and adaptive decision process. Our key contributions include: \bullet A reasoning-aware instruction tuning framework that reformulates diverse recommendation tasks, including sequential recommendation, direct recommendation, CTR prediction, and explanation generation, into a unified chain-of-thought (CoT) format, enabling the VLM to verbalize intermediate reasoning steps and improve task alignment. \bullet An evidence-horizon curriculum learning strategy that gradually expands the complexity of reasoning chains by controlling user-item sparsity levels during training. This approach enhances generalization in cold-start and long-tail scenarios. \bullet An uncertainty-guided tool delegation mechanism that equips the agent with the ability to assess its the query risk and dynamically invoke lightweight classical models, balancing computational cost and predictive robustness. \bullet Extensive empirical validation across five public benchmarks and four recommendation tasks, demonstrating that ReasonRec achieves over 30% improvement in HR@5 and NDCG@5 compared to prior state-of-the-art models; adapts effectively to cold-start, time-shift, and multi-domain scenarios; and reduces inference cost by more than 30% through selective delegation. 2 Related Work Generative models for recommendation systems. Recent advances in recommendation systems have witnessed a paradigm shift toward generative architectures and multimodal content understanding. Building upon the foundation of large language models (LLMs) Li et al. 2023c , pioneering works like Transformers4Rec de Souza Pereira Moreira et al. 2021 and BERT4Rec Sun et al. 2019 employ Transformer architectures for sequential modeling, establishing frameworks for transferable recommendation through language model pretraining. Subsequent innovations extend this paradigm through diverse representation learning

Read the complete text