MDVT: Enhancing Multimodal Recommendation with Model-Agnostic Multimodal-Driven Virtual Triplets Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Disc
Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, Yijie Li, Mengran Li, Puzhen Wu, Edith C. H. Ngai · 2025
Read MDVT: Enhancing Multimodal Recommendation with Model-Agnostic Multimodal-Driven Virtual Triplets Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Disc by Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, Yijie Li, Mengran Li, Puzhen Wu, Edith C. H. Ngai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16665.txt.
Introduction
Jinfeng Xu and co-authors present a 2025 method for sharpening recommendation systems that use text and images. Sparse user histories starve such systems, and past work treats the extra media as mere side data. Their method, MDVT, builds virtual triplets from the media itself: guessed user-item links that act as extra training signal, whatever base model sits below.
Three warm-up rules decide when the guessed links are ripe to use: a fixed count, a loss-driven trigger, or a blend of both. A tuned pairwise loss then trains the model jointly without skewing its gradients. On several real data sets, adding the method lifted accuracy, most of all where data ran sparsest.
Catalog record
| CREATOR | Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, Yijie Li, Mengran Li, Puzhen Wu, Edith C. H. Ngai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16665
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MDVT: Enhancing Multimodal Recommendation with Model-Agnostic Multimodal-Driven Virtual Triplets Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD ’25), August 3–7, 2025, Toronto, ON, Canada DOI: 10.1145/3711896.3737042 ISBN: 979-8-4007-1454-2/2025/08 CCS: Information systems Recommender systems; Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, Yijie Li, Mengran Li, Puzhen Wu, Edith C. H. Ngai Abstract . The data sparsity problem significantly hinders the performance of recommender systems, as traditional models rely on limited historical interactions to learn user preferences and item properties. While incorporating multimodal information can explicitly represent these preferences and properties, existing works often use it only as side information, failing to fully leverage its potential. In this paper, we propose MDVT, a model-agnostic approach that constructs multimodal-driven virtual triplets to provide valuable supervision signals, effectively mitigating the data sparsity problem in multimodal recommendation systems. To ensure high-quality virtual triplets, we introduce three tailored warm-up threshold strategies: static, dynamic, and hybrid. The static warm-up threshold strategy exhaustively searches for the optimal number of warm-up epochs but is time-consuming and computationally intensive. The dynamic warm-up threshold strategy adjusts the warm-up period based on loss trends, improving efficiency but potentially missing optimal performance. The hybrid strategy combines both, using the dynamic strategy to find the approximate optimal number of warm-up epochs and then refining it with the static strategy in a narrow hyper-parameter space. Once the warm-up threshold is satisfied, the virtual triplets are used for joint model optimization by our enhanced pair-wise loss function without causing significant gradient skew. Extensive experiments on multiple real-world datasets demonstrate that integrating MDVT into advanced multimodal recommendation models effectively alleviates the data sparsity problem and improves recommendation performance, particularly in sparse data scenarios. 1. Introduction The rapid development of the internet has led to an information explosion, making recommender systems indispensable for navigating vast amounts of data. Traditional recommender systems rely on modeling user preferences through historical user-item interactions ( He et al. 2020 ; Xu et al. 2024b ; Xu et al. 2024a ; Chen et al. 2025b ) . However, the data sparsity problem significantly hinders the performance of these systems, as they depend solely on limited historical interactions to implicitly learn user preferences and item properties. Incorporating multimodal information ( Chen et al. 2024 ; Lin et al. 2025 ) —such as images and textual descriptions—allows for explicit representation of user preferences and item properties, potentially alleviating the data sparsity problem. Several recent works ( He and McAuley 2016 ; Chen et al. 2017 ; Xu et al. 2025c ) have integrated multimodal content into recommendation models. For example, VBPR ( He and McAuley 2016 ) extends the matrix factorization framework to incorporate item visual features, while ACF ( Chen et al. 2017 ) introduces a hierarchically structured attention network to capture user preferences at the component level. Graph Convolutional Networks (GCNs) have also gained attention in this context ( Guo et al. 2024 ; Wei et al. 2019 ; Wei et al. 2020 ; Zhang et al. 2021 ; Zhou and Shen 2023 ) . Models like MMGCN ( Wei et al. 2019 ) and GRCN ( Wei et al. 2020 ) employ GCNs to integrate multimodal information into the message-passing process, enhancing the inference of user and item representations. To further exploit the rich multimodal information, LATTICE ( Zhang et al. 2021 ) and FREEDOM ( Zhou and Shen 2023 ) construct item-item graphs to aggregate semantically similar items. LGMRec ( Guo et al. 2024 ) utilizes hyper-graph structures to learn both global and local representations, capturing complex relationships in multimodal information. However, existing works typically use multimodal information only as side information to enhance the learning of user preferences, failing to fully leverage its potential. They primarily focus on improving item representations using multimodal content, while user representations are still learned solely from historical interactions. This limitation becomes more pronounced in data sparsity scenarios, where users have limited interaction records. We propose that the similarity between user and item modality representations can serve as valuable supervision signals beyond explicit user-item interactions. To leverage this insight, we introduce M ultimodal- D riven V irtual T riplets (MDVT), a novel, model-agnostic approach that constructs virtual triplets based on multimodal information. These virtual triplets provide informative supervision signals, effectively mitigating the data sparsity problem in multimodal recommendation systems. A key challenge is that, unlike items, users do not have inherent multimodal information in recommendation scenarios. Users’ modality representations must be learned from scratch by initializing embeddings randomly and refining them through model optimization. Consequently, the initial similarity between user and item modality representations may not provide high-quality supervision signals. To address this, we introduce three tailored warm-up threshold strategies: • Static Warm-up Threshold Strategy : This strategy exhaustively searches for the optimal number of warm-up epochs, ensuring that user modality representations are sufficiently learned before constructing virtual triplets. While effective, it is time-consuming and computationally intensive due to the thorough hyper-parameter tuning required. • Dynamic Warm-up Threshold Strategy : This strategy adjusts the warm-up period based on the trend of loss changes during training. It improves efficiency by reducing the need for extensive hyper-parameter tuning, automatically determining when user representations are adequately learned. However, it may not always find the optimal number of warm-up epochs compared to the static strategy. • Hybrid Warm-up Threshold Strategy : Combining the strengths of both strategies, the hybrid strategy first employs the dynamic strategy to find the approximate optimal number of warm-up epochs and then applies the static strategy within a narrow hyper-parameter space. This allows for efficient training with a balance between computational cost and performance optimization. Once the warm-up threshold is satisfied, the virtual triplets are used for joint model optimization through our enhanced pair-wise loss function, enhancing the learning process without causing significant gradient skew ( Li et al. 2018 ; Yu et al. 2020 ) . Our MDVT approach is plug-and-play and can be easily integrated into any existing multimodal recommendation model, improving their performance, particularly in data sparsity scenarios. To validate the effectiveness of MDVT, we conducted extensive experiments on multiple real-world datasets adopting various advanced multimodal recommendation models. The results demonstrate that integrating MDVT into these models significantly alleviates the data sparsity problem and improves recommendation performance, especially for users with limited interaction records. Additionally, we would like to highlight the key distinction between our work and prior studies. Our virtual triplets are constructed based on the similarity between dynamically learned user and item representations, which are better aligned with the recommendation task with a sufficient warm-up phase. In contrast, prior works typically construct virtual samples based on the similarity of raw features between items. 2. Preliminary In this section, we provide an overview of graph collaborative filtering (GCF), the common paradigm of advanced multimodal recommendations, which adopts graph neural network (GNN) into collaborative filtering (CF) with multimodal features. CF tasks usually contains a user set \mathcal{U}=\{u_{1},...,u_{|\mathcal{U}|}\} and an item set \mathcal{I}=\{i_{1},...,i_{|\mathcal{I}|}\} . In multimodal scenarios, each item contains multiple features, we introduce modality-specific item embedding i^{m} for each item i