Semantics Meet Signals: Dual Codebook Representationl Learning for Generative Recommendation

Zheng Hui, Xiaokai Wei, Reza Shirkavand, Chen Wang, Weizhi Zhang, Alejandro Peláez, Michelle Gong · 2025

Read Semantics Meet Signals: Dual Codebook Representationl Learning for Generative Recommendation by Zheng Hui, Xiaokai Wei, Reza Shirkavand, Chen Wang, Weizhi Zhang, Alejandro Peláez, Michelle Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20673.txt.

Introduction

Zheng Hui and co-authors address in 2025 a quiet inefficiency inside generative recommender systems, which suggest products by generating item codes token by token. Each item is stored as tokens from a single codebook, a dictionary of codes, used for every item. Popular items carry rich behaviour records, while obscure long-tail items depend mostly on what they are, so one codebook serves both badly.

Their system, FlexCode, splits a fixed token budget across two codebooks. One captures collaborative signals, patterns in shared user choices; the other captures semantic content. A small mixture-of-experts gate shifts the balance per item, favouring behaviour tokens for hits and meaning tokens for rare items. On public and industrial-scale datasets FlexCode beat strong baselines on every ranking metric used, and even a fixed half-and-half version outperformed most rivals, with the clearest gains on hard-to-recommend tail items.

Catalog record

CREATORZheng Hui, Xiaokai Wei, Reza Shirkavand, Chen Wang, Weizhi Zhang, Alejandro Peláez, Michelle Gong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-20673

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semantics Meet Signals: Dual Codebook Representationl Learning for Generative Recommendation Zheng Hui, Xiaokai Wei, Reza Shirkavand, Chen Wang, Weizhi Zhang, Alejandro Peláez, Michelle Gong Abstract Generative recommendation has recently emerged as a powerful paradigm that unifies retrieval and generation, representing items as discrete semantic tokens and enabling flexible sequence modeling with autoregressive models. Despite its success, existing approaches rely on a single, uniform codebook to encode all items, overlooking the inherent imbalance between popular items rich in collaborative signals and long-tail items that depend on semantic understanding. We argue that this uniform treatment limits representational efficiency and hinders generalization. To address this, we introduce FlexCode, a popularity-aware framework that adaptively allocates a fixed token budget between a collaborative filtering (CF) codebook and a semantic codebook. A lightweight MoE dynamically balances CF-specific precision and semantic generalization, while an alignment and smoothness objective maintains coherence across the popularity spectrum. We perform experiments on both public and industrial-scale datasets, showing that FlexCode consistently outperform strong baselines. FlexCode provides a new mechanism for token representation in generative recommenders, achieving stronger accuracy and tail robustness, and offering a new perspective on balancing memorization and generalization in token-based recommendation models. 1 Introduction Recommender systems are a central component of modern information platforms, enabling personalized ranking and retrieval in e-commerce, streaming media, and social feeds Gomez-Uribe and Hunt 2016 ; Wang et al. 2025 ; Hui et al. 2025 . Traditional recommendation methods Koren et al. 2009 ; He et al. 2017 ; Sun et al. 2019 ; Geng et al. 2022 learn latent representations from observed user–item interactions. These models are effective for frequently interacted (head) items because they are able to memorize fine-grained co-occurrence structure. However, they exhibit systematic degradation on long-tail and cold-start items, where interaction data is sparse Zhang et al. 2021 ; Liu et al. 2023 . This reflects an inherent memorization bias: collaborative filtering captures relational specificity but lacks semantic generalization. Semantic-ID based Generative recommendation Rajput et al. 2023 ; Wang et al. 2024 has recently emerged as a compelling alternative paradigm. Rather than treating recommendation as score prediction over fixed item IDs, generative recommenders map items into discrete semantic tokens using vector quantization or learned codebooks, and then train an autoregressive model to generate future items in sequence. This formulation offers several advantages. It unifies recommendation with sequence generation, making it compatible with large language models and multilingual or multimodal supervision. It also supports better generalization to unseen items, since tokenization can draw on textual, visual, or metadata semantics rather than relying solely on collaborative history Lv et al. 2024 . Despite these benefits, current generative recommenders suffer from two fundamental limitations that prevent them from fully replacing traditional CF systems in practice. The first limitation is representation entanglement Shirkavand et al. 2025 ; Fang et al. 2025 . Most existing approaches adopt a single, shared codebook for all items, and attempt to embed both semantic content information and collaborative interaction information into the same quantized space. This forces the model to compress heterogeneous factors of variation into a single representation. As a consequence, the learned item tokens are often neither semantically pure nor collaboration-accurate. The same codeword is expected to simultaneously reflect textual or visual meaning and encode high-order co-consumption structure. This coupling leads to interference. For head items, the semantic signal can dilute highly localized collaborative patterns. For tail items, the collaborative signal can dominate even when it is statistically unreliable, which harms cold-start behavior. In short, entanglement in a single codebook induces representational collapse: neither factor is modeled optimally Baykal et al. 2024 . The second limitation is static capacity allocation Zhang et al. 2024 . Existing semantic identifier and unified ID-plus-semantic methods allocate a fixed representational budget to each item. Every item is assigned the same number and type of codebook tokens, regardless of its popularity and data regime. This is in tension with the long-tailed nature of recommender data Klimashevskaia et al. 2024 . Head items have abundant interaction data and therefore benefit from high-capacity collaborative representations. Tail items, on the other hand, lack interaction support and rely more strongly on semantic evidence such as text descriptions, visual attributes, or structured metadata. Treating these two classes of items identically wastes capacity in one regime and starves it in the other. In particular, it leads to overfitting on the head while under-representing the tail. We view these two limitations as a more general problem in generative recommendation: adaptive capacity allocation. Given a fixed token budget per item, how should a system decide how much of that capacity to devote to collaborative specificity versus semantic generalization, and how can it avoid destructive interference between the two? To address these challenges, we propose FlexCode , a popularity-aware generative recommendation framework that explicitly disentangles collaborative and semantic channels and dynamically allocates representational capacity between them. FlexCode introduces two separate codebooks: a collaborative codebook \mathbf{C}_{\mathrm{CF}} that captures high-order interaction structure, and a semantic codebook \mathbf{C}_{\mathrm{SEM}} that captures modality-grounded meaning. Our framework uses a lightweight mixture-of-experts (MoE) gating mechanism that conditions on item statistics such as popularity or interaction sparsity, and selects how much capacity to allocate to each codebook under a fixed overall token budget. Intuitively, head items are routed toward collaborative experts and receive more tokens from \mathbf{C}_{\mathrm{CF}} , while tail items are routed toward semantic experts and receive more tokens from \mathbf{C}_{\mathrm{SEM}} . In addition to disentangling representation sources and adapting capacity, FlexCode introduces two regularization objectives to ensure stability and coherence. Together, these components allow FlexCode to interpolate smoothly between collaborative-style and semantic-style item encodings, rather than forcing a hard dichotomy. Conceptually, this approach operationalizes the intuition that memorization and generalization are not mutually exclusive, but should be emphasized differently for different regions of the item distribution. We evaluate FlexCode on both publicly available benchmarks and large-scale industrial datasets. Across all settings, our method outperforms Item-ID based, Semantic-ID and unified representation baselines on standard ranking metrics. Our main contributions are as follows: • We identify two structural limitations of existing generative recommenders: representation entanglement in a single shared codebook and static capacity allocation that ignores item popularity and data sparsity. We formalize these limitations as instances of an adaptive capacity allocation problem. • We propose FlexCode , a framework that factorizes item representations into a collaborative codebook and a semantic codebook, and uses a popularity-aware mixture-of-experts gate to allocate a fixed token budget between them on a per-item basis. • We demonstrate consistent improvements over recent Semantic-ID and unified-ID baselines on both public and industrial datasets, and show that FlexCode yields better head–tail balance and superior robustness under tight token budgets. Figure 1: Overview of the FlexCode framework for generative recommendation. Each item is encoded by a dual codebook with collaborative and semantic codebooks, aligned via a cross-codebook contrastive objective. A popularity-aware Mixture-of-Experts (MoE) router adaptively allocates the budget between codebooks, and an autoregressive Transformer is trained on the resulting sequences to generate items. 2 Background and Preliminaries 2.1 Generative Recommendation Framework Let \mathcal{U} denote the set of users and \mathcal{I} the set of items. For each user u\in\mathcal{U} , we observe an interaction sequence \mathbf{s}_{u}=[i_{1},i_{2},\ldots,i_{T}], where i_{t}\in\mathcal{I} represents the t -th item interacted with by user

Read the complete text