Cold-Starts in Generative Recommendation: A Reproducibility Study 410 Conference: 49th International ACM SIGIR Conference on Research and Development in Information Retrieval,; July 20-24, 2026; Melbourne, Australia CCS: Information systems Recommender systems
Zhen Zhang, Jujia Zhao, Xinyu Ma, Xin Xin, Maarten de Rijke, Zhaochun Ren · 2026
Read Cold-Starts in Generative Recommendation: A Reproducibility Study 410 Conference: 49th International ACM SIGIR Conference on Research and Development in Information Retrieval,; July 20-24, 2026; Melbourne, Australia CCS: Information systems Recommender systems by Zhen Zhang, Jujia Zhao, Xinyu Ma, Xin Xin, Maarten de Rijke, Zhaochun Ren (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29845.txt.
Introduction
Zhang and co-authors run a reproducibility study of generative recommenders under cold-start conditions. New users and new items both lack interaction history. Language-model recommenders are often expected to help via item text and test-time context. Yet cold-start is rarely the main test setting in past work. Design choices also tend to change together, muddying reported gains. Their study uses one unified suite of cold-start protocols. It aims to make such gains readable at last.
Catalog record
| CREATOR | Zhen Zhang, Jujia Zhao, Xinyu Ma, Xin Xin, Maarten de Rijke, Zhaochun Ren |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49413 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.29845
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cold-Starts in Generative Recommendation: A Reproducibility Study 410 Conference: 49th International ACM SIGIR Conference on Research and Development in Information Retrieval,; July 20-24, 2026; Melbourne, Australia CCS: Information systems Recommender systems Zhen Zhang, Jujia Zhao, Xinyu Ma, Xin Xin, Maarten de Rijke, Zhaochun Ren Abstract . Cold-start recommendation remains a central challenge in dynamic, open-world platforms, requiring models to recommend for newly registered users (user cold-start) and to recommend newly introduced items to existing users (item cold-start) under sparse or missing interaction signals. Recent generative recommenders built on pre-trained language models (PLMs) are often expected to mitigate cold-start by using item semantic information (e.g., titles and descriptions) and test-time conditioning on limited user context. However, cold-start is rarely treated as a primary evaluation setting in existing studies, and reported gains are difficult to interpret because key design choices, such as model scale, identifier design, and training strategy, are frequently changed together. In this work, we present a systematic reproducibility study of generative recommendation under a unified suite of cold-start protocols. We reproduce representative generative recommenders and evaluate their efficacy across both user and item cold-start settings. Crucially, we isolate the impact of three critical design dimensions: (i) model scale, (ii) item identifier design (e.g., atomic, semantic identifiers), and (iii) training strategy (e.g., supervised fine-tuning, reinforcement learning). Through a comprehensive evaluation on three datasets, we uncover some key findings. First, scaling up model size provides only marginal improvements and does not fundamentally close the cold-start gap. Second, identifier design plays a decisive role: textual identifiers can substantially improve item cold-start but introduce clear trade-offs by harming warm and user cold-start performance, while more compositional semantic coding schemes show better robustness. Third, reinforcement learning does not consistently improve and can even degrade performance under cold-start. Our study provides empirical evidence and practical guidance for enhancing the generalization capabilities of generative recommendation systems in cold-start scenarios. 1 1 1 Our data and code is available at https://anonymous.4open.science/r/ColdGenrec-0DEC † † cc-license: by 1. Introduction Recommender systems are a core component of two-sided marketplaces, matching users with items in dynamic, open-world environments ( Fu et al. 2026 ; Lin et al. 2025 ) . A fundamental challenge in these settings is cold-start on both sides: recommending relevant items to newly registered users (user cold-start) and exposing newly introduced items to existing users (item cold-start) ( Wei et al. 2021 ; Huang et al. 2025 ) . In these data-sparse scenarios, initial recommendations dictate long-term user retention, while early exposure determines whether new items can acquire the feedback necessary for model convergence. Consequently, delivering effective recommendations under both user and item cold-start conditions remains a central challenge for building robust recommender systems. Most traditional recommender models rely heavily on historical interactions and therefore perform poorly when either users or items lack reliable feedback signals ( Ding et al. 2024 ) . To address cold-starts, these methods typically incorporate auxiliary mechanisms, such as using side information, data augmentation, or meta-learning, to construct or adapt representations for new users and items ( Monteil et al. 2024 ; Zhang et al. 2022 ; Vartak et al. 2017 ) . More recently, generative recommendation built on pre-trained language models (PLMs) has emerged as an alternative paradigm and is often expected to mitigate cold-starts for both the user and item sides ( Zhang et al. 2025a ) . Figure 1 illustrates a general pipeline of generative recommendation. Figure 1. Generative recommendation pipeline. For user cold-starts, PLMs can use test-time conditioning to infer user preferences from limited interaction context, thereby enabling generalization to previously unseen or sparsely observed users. For item cold-starts, pre-trained knowledge can help interpret the semantic content of newly introduced items through their identifiers (e.g., textual descriptors or semantic codes) ( Tan et al. 2024 ; Rajput et al. 2023 ) . 1.1. Cold-starts in generative recommendation Despite these hypotheses, cold-starts are seldom treated as a primary evaluation setting for generative recommendation, leaving open how these models behave when users or items have little to no interaction history. Moreover, existing generative recommendation methods often conflate multiple design factors, such as model scale, identifier design, and training strategy, which makes factor-wise attribution difficult. Even when performance differences are observed, it remains unclear which factors are responsible. 1.2. Research questions This gap motivates a systematic reproducibility study that isolates and evaluates the effects of key confounding factors under a unified set of cold-start settings. Accordingly, we conduct a reproducibility study guided by four research questions: (i) RQ1 : How do representative generative recommenders perform under cold-start settings? (ii) RQ2 : How does model scale influence cold-start performance in generative recommendation? (iii) RQ3 : How does identifier design (e.g., atomic, semantic identifiers) affect the ability of generative models to generalize to unseen users and items? (iv) RQ4 : How do different training strategies (e.g., supervised fine-tuning, or reinforcement learning) impact cold-start behavior? 1.3. Findings Our experiments provide the following key findings in response to the above research questions: (i) The cold-start problem exhibits highly asymmetric behavior: while user cold-starts lead to only moderate performance degradation, item cold-starts remain substantially more challenging for existing generative recommendation approaches. In many cases, performance drops are dramatic when moving from warm-test to item cold-test. (ii) We observe a scaling trend, whereas the gains are marginal. Increasing model size yields only limited relative improvements under cold-start settings, while the performance gap between warm and item cold-start remains large. This suggests that scaling alone is insufficient to bridge the cold-start gap. (iii) Identifier design plays a critical role in item cold-start performance and reveals a pronounced trade-off. Textual identifiers can substantially improve item cold-start behavior, often yielding several-fold gains over semantic-code or atomic identifiers. However, these improvements come at the cost of noticeably degraded performance in warm-test and user cold-test settings. This pattern highlights an inherent tension between semantic generalization and identifier discriminability. We further find that compositional semantic coding (e.g., OPQ-style codes) alleviates the item cold-start collapse without sacrificing warm-test accuracy, suggesting that reducing strong sequential dependence in identifier prediction may improve cold-start robustness. (iv) Reinforcement learning does not consistently improve cold-start performance and can even degrade robustness. This indicates that reward objectives optimized on the training distribution may not transfer effectively under cold-start distribution shifts, requiring careful objective design to avoid exacerbating generalization gaps. 1.4. Contributions Our main contributions are as follows: (i) We conduct a comprehensive evaluation of representative generative recommenders under a unified suite of user and item cold-start settings, providing a controlled assessment of their cold-start behavior. (ii) We establish a reproducible cold-start evaluation framework that isolates key design factors, including model scale, identifier design, and training strategy, enabling systematic and factor-wise analysis of generative recommenders under controlled protocols. (iii) We offer actionable insights for enhancing the generalization capabilities of generative recommendation systems in cold-start scenarios. 2 2 2 Following ACM’s terminology, our study is a replicability study (“different team, different setup”). For all our research questions we focus on a specific experimental setup, viz. that of user and item cold-starts, that was not the core focus of the papers that we reproduce and can therefore be considered to be a different setup. 2. Related Work 2.1. Generative retrieval With the rapid advancement