SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation

Wei Chen, Xingyu Guo, Shuang Li, Fuwei Zhang, Meng Yuan, Jing Fan, Zhao Zhang, Deqing Wang, Fuzhen Zhuang · 2026

Read SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation by Wei Chen, Xingyu Guo, Shuang Li, Fuwei Zhang, Meng Yuan, Jing Fan, Zhao Zhang, Deqing Wang, Fuzhen Zhuang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18920.txt.

Introduction

Wei Chen and eight colleagues improve generative recommendation by using synergy between data types rather than merely aligning them. Current systems produce item codes and fuse signals from each type, but they miss traits that no single type reveals alone. Their SynGR model rewards links across types during generation and curbs leaning on dominant types. It thus captures item meaning beyond shared or single-type signals, and tests on three benchmarks show it beating prior methods.

Catalog record

CREATORWei Chen, Xingyu Guo, Shuang Li, Fuwei Zhang, Meng Yuan, Jing Fan, Zhao Zhang, Deqing Wang, Fuzhen Zhuang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55335
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18920

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation Wei Chen, Xingyu Guo, Shuang Li, Fuwei Zhang, Meng Yuan, Jing Fan, Zhao Zhang, Deqing Wang, Fuzhen Zhuang Abstract Generative Recommendation (GR) has emerged as a promising paradigm by formulating item recommendation as a sequence-to-sequence generation task over item identifiers. Recent studies have incorporated multimodal signals to provide richer token-level evidence for generation. However, existing approaches largely rely on alignment-centric fusion and underexplore synergistic information across modalities. In practice, synergistic information plays a critical role in capturing emergent item properties that cannot be inferred from any single modality alone. Such properties encode intrinsic item semantics and guide user preferences, enabling models to move beyond surface-level feature matching. To address this limitation, we propose SynGR , a synergistic generative recommendation framework that explicitly encourages the exploitation of cross-modal dependencies during generation. By constraining overreliance on dominant modalities, SynGR enables the model to capture emergent item semantics beyond shared or modality-specific signals. Extensive experiments across three benchmark datasets demonstrate that SynGR achieves superior performance. Code is available at: SynGR . † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : (a) Illustration of cross-modal information decomposition, where \textbf{S},\textbf{R},\mathbf{U}_{\mathbf{t}} and \mathbf{U}_{\mathbf{v}} denote synergistic , redundant , and modality-specific unique information, respectively. (b) Comparison of synergistic components estimated using a normalized PID-inspired performance decomposition ( 21 ) . (c) The distribution of visual and textual attention across datasets. The evolution of recommender systems has recently shifted from traditional discriminative ranking to Generative Recommendation (GR) ( 20 ; 18 ; 65 ; 6 ; 7 ) . Unlike conventional methods that map users and items to latent embeddings for heuristic scoring, GR formulates user behavior modeling as a sequence-to-sequence generation task ( 25 ; 26 ) . By representing items with discrete semantic identifiers, such as quantized tokens or hierarchical IDs ( 41 ) , these frameworks utilize the sequential reasoning power of Pre-trained Language Models (PLMs) to capture complex and evolving user preferences ( 15 ; 12 ; 58 ; 17 ) . Recent studies ( 35 ; 52 ) in GR have begun to incorporate multimodal signals, such as visual and textual information, to provide richer item evidence and improve the generation of discrete semantic identifiers. However, the practical gains of these approaches remain limited. Most existing methods ( 70 ; 59 ; 62 ) rely on alignment-centric fusion strategies that enforce cross-modal consistency through shared representations. While such designs are effective for basic semantic matching, they primarily encourage modality agreement rather than modality interaction, and consequently struggle to capture intricate synergistic information that arises only from the joint interplay across modalities. Figure 1 (a) illustrates the critical role of synergistic information \mathbf{S} in capturing emergent item properties that are essential for accurate recommendation. Taking a luxury handbag as an example, the visual modality \mathbf{U}_{\mathbf{v}} captures aesthetic nuances such as quilted leather textures and fine-grained embellishments, while the textual modality \mathbf{U}_{\mathbf{t}} provides explicit signals like brand identity and pricing. Individually, neither modality is sufficient to characterize the item’s full semantic profile. Instead, their synergy \mathbf{S} facilitates the emergence of higher-level attributes, such as brand prestige and luxury positioning, which are unattainable from any single modality alone. These emergent properties constitute the intrinsic semantics that shape user preferences, enabling the model to move beyond surface-level feature matching toward a deeper understanding of user intent. Despite its importance, the synergistic component \mathbf{S} remains insufficiently explored in existing generative paradigms, which often emphasize redundant or modality-unique signals. As an illustrative case, our empirical analysis in Figure 1 (b) reveals that even a representative state-of-the-art method, MACRec ( 62 ) , captures a mere 7.5% of its learned representation as synergistic information on the Arts dataset 1 1 1 The calculation details are shown in Appendix C . . This observation suggests a systemic challenge in current multimodal GR: the inherent difficulty of effectively unleashing the latent potential of cross-modal synergy . This limitation stems from inherent disparities between modalities under the generative training objective. Textual inputs are discrete, sequential, and semantically compact, with individual tokens often encoding highly discriminative signals such as brand, category, or style, whereas visual embeddings are continuous, high-dimensional, and noisy, requiring significant aggregation to manifest coherent semantics ( 40 ) . When such heterogeneous modalities are mapped into codebooks of comparable capacity, the fixed codebook budget represents fundamentally unequal semantic depth, leading to a mismatch in effective information density. In recommendation scenarios, this imbalance allows certain modalities to provide a more direct and reliable mapping to the target identifier, forming a path of least resistance for minimizing the generation loss ( 34 ) . Consistent with this behavior, Figure 1 (c) reveals that across multiple datasets, the current GR model assigns disproportionately higher attention weights to the textual modality, biasing learning toward text-dominant representations during next-token prediction. Consequently, complementary evidence from secondary modalities is progressively deemphasized. Since synergistic information requires high-order interactions, this unimodal convergence suppresses the emergence of complex multimodal patterns. Motivated by these observations, we argue that bridging the synergy gap requires explicit intervention to prevent generative models from collapsing into unimodal shortcuts. To this end, we propose SynGR , a synergistic generative recommendation framework that actively disrupts the path of least resistance during training. The key idea is to constrain the model’s reliance on dominant modalities, thereby compelling it to leverage higher-order cross-modal dependencies to satisfy the generative objective. SynGR achieves this through a saliency-aware masking mechanism that adaptively attenuates dominant signals and encourages the exploration of latent synergistic information across modalities. To further stabilize and refine these emergent semantics, the framework incorporates a synergy-oriented contrastive objective that explicitly promotes cross-modal interaction. Our main contributions are summarized as follows: (i) Insight. We identify a synergy gap in generative recommendation, showing that information density mismatch across modalities induces unimodal shortcuts and suppresses higher-order cross-modal semantics. (ii) Method. We propose SynGR, a synergistic generative recommendation framework that explicitly disrupts unimodal shortcuts and compels the model to leverage cross-modal dependencies during generation. (iii) Evaluation. We demonstrate through extensive experiments that SynGR consistently enhances synergistic learning and improves recommendation performance across three real-world datasets, achieving average gains of around 10% over strong baselines. 2 Related Work Sequential Recommendation. Sequential recommendation ( 3 ) aims to capture the evolution of user interests from historical interaction logs. Early research predominantly focused on temporal dependencies using Recurrent Neural Networks (RNNs) ( 37 ) , with pioneering models such as GRU4Rec ( 13 ) and NARM ( 24 ) establishing the foundation for session-based modeling. To address the limitations of RNNs in capturing long-range dependencies, SASRec ( 19 ) introduced self-attention mechanisms, while BERT4Rec ( 45 ) employed a bidirectional Cloze objective to enhance sequence representations. This field has recently shifted toward a unified paradigm where recommendation is treated as a language modeling task. For instance, P5 ( 9 ; 46 ) leveraged PLMs to convert diverse recommendation tasks into a standardized text-to-text format. To further alleviate data sparsity, researchers extended this paradigm to the multimodal domain. Notably, VIP5 ( 10 ) served as a multimodal foundation model by integrating visual and textual signals into the generative framework. Generative Recommendation. The advent of Large Language Models (LLMs)

Read the complete text