FITMM: Adaptive Frequency-Aware Multimodal Recommendation via Information-Theoretic Representation Learning Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25)
Wei Yang, Rui Zhong, Yiqun Chen, Shixuan Li, Heng Ping, Chi Lu, Peng Jiang · 2026
Read FITMM: Adaptive Frequency-Aware Multimodal Recommendation via Information-Theoretic Representation Learning Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25) by Wei Yang, Rui Zhong, Yiqun Chen, Shixuan Li, Heng Ping, Chi Lu, Peng Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22498.txt.
Introduction
Wei Yang and six co-authors sharpen product suggestions by reading items in the frequency domain. Modern recommenders blend item images and text to guess what a user wants. Their 2026 preprint, FITMM, notes that most blends work in plain space, which can blur signal shape and add clash and waste.
Their method learns shared codes with information theory as its guide, tuned to each frequency band. A cleaner blend should guess taste more truly. It may also explain its picks in a clearer way. The work fits shops, media, and feeds. It is a fresh angle on a daily tool.
Catalog record
| CREATOR | Wei Yang, Rui Zhong, Yiqun Chen, Shixuan Li, Heng Ping, Chi Lu, Peng Jiang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22498
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FITMM: Adaptive Frequency-Aware Multimodal Recommendation via Information-Theoretic Representation Learning Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland DOI: 10.1145/3746027.3755540 ISBN: 979-8-4007-2035-2/2025/10 CCS: Information systems Recommender systems CCS: Information systems Multimedia and multimodal retrieval Wei Yang, Rui Zhong, Yiqun Chen, Shixuan Li, Heng Ping, Chi Lu, Peng Jiang Abstract . Multimodal recommendation aims to enhance user preference modeling by leveraging rich item content such as images and text. Yet dominant systems fuse modalities in the spatial domain, obscuring the frequency structure of signals and amplifying misalignment and redundancy. We adopt a spectral information-theoretic view and show that, under an orthogonal transform that approximately block-diagonalizes bandwise covariances, the Gaussian Information Bottleneck objective decouples across frequency bands, providing a principled basis for separate-then-fuse paradigm. Building on this foundation, we propose FITMM , a Frequency-aware Information-Theoretic framework for multimodal recommendation. FITMM constructs graph-enhanced item representations, performs modality-wise spectral decomposition to obtain orthogonal bands, and forms lightweight within-band multimodal components. A residual, task-adaptive gate aggregates bands into the final representation. To control redundancy and improve generalization, we regularize training with a frequency-domain IB term that allocates capacity across bands (Wiener-like shrinkage with shut-off of weak bands). We further introduce a cross-modal spectral consistency loss that aligns modalities within each band. The model is jointly optimized with the standard recommendation loss. Extensive experiments on three real-world datasets demonstrate that FITMM consistently and significantly outperforms advanced baselines. The source code is available at: https://github.com/llm-ml/FITMM . 1. INTRODUCTION With the surge of multimodal content in e-commerce, short video platforms, and social media, users are increasingly exposed to items described through diverse modalities such as product images, textual descriptions, videos, and reviews ( Han et al. 2022 ; Cen et al. 2020 ; He et al. 2021 ; Zhou et al. 2023b ; Yang et al. 2024 ) . This has led to the emergence of Multi-Modal Recommender Systems (MMRS), which aim to leverage heterogeneous sources to better capture user preferences and improve recommendation performance ( Wei et al. 2019 ; Deldjoo et al. 2020 ; Liu et al. 2024a ; Malitesta et al. 2024 ; Yang et al. 2025b ) . Effectively modeling and integrating such multimodal information has become a central challenge in modern recommender system research ( Wang et al. 2023 ; Liu et al. 2024c ; Li et al. 2025a ; Liu et al. 2025 ; Xu et al. 2025 ; Zhao et al. 2025 ; Gu et al. 2025 ) . Recent advances have centered on Graph Neural Networks (GNNs) to capture the high-order collaborative signals in user-item graphs ( Liu et al. 2022 ; Guo et al. 2024 ; Zhou et al. 2023c ; Li et al. 2024a ; Kim et al. 2024 ) . The dominant paradigm, however, involves a critical flaw: after encoding, modalities are fused in the spatial domain using simple gating or attention mechanisms. This approach struggles with the inherent statistical discrepancies and semantic misalignment across heterogeneous modalities (e.g., image vs. text) ( Yang et al. 2023a ; Shang et al. 2024 ) . By forcing a direct projection into a shared space, these methods often lead to one modality’s signal overpowering another or critical information being diluted during fusion ( Li et al. 2024b ; Yi and Ounis 2024 ; Bai et al. 2024 ) . More fundamentally, this paradigm overlooks the intrinsic frequency domain structure of multimodal signals. Information is not monolithic; it exists on a spectrum where different frequencies carry distinct semantic weight. Low-frequency components typically encode stable, cross-modal semantics (like general categories or style), whereas high-frequency components capture the fine-grained, modality specific details that are crucial for personalization ( Ong and Khong 2024 ; Du et al. 2023 ) . Current models, by fusing signals in the spatial domain, indiscriminately mix these components, leading to over-smoothing where dominant low-frequency signals drown out valuable high-frequency nuances. This lack of spectral awareness points to a deeper information-theoretic failure. An ideal model should not attempt to preserve all information, but rather isolate and retain only the most predictive frequency bands for the recommendation task. Without a principled mechanism to disentangle and filter information, current models create representations that are bloated with redundancy from low-frequency components and corrupted by noise from high-frequency ones. This compromises the model’s expressive power and generalization ability, underscoring the need for a new, theoretically-grounded approach. To address these limitations, we present FITMM , a frequency-aware information-theoretic framework for multimodal recommendation. Our approach is grounded in a spectral view: an orthogonal or tight-frame transform approximately block-diagonalizes bandwise covariances, and the Gaussian Information Bottleneck (IB) decouples across frequency bands. On this foundation, FITMM instantiates a principled “separate–then–fuse” pipeline. Concretely, we first build graph-enhanced representations for ID, visual, and textual signals and perform modality-wise spectral decomposition to obtain orthogonal bands. Within each band, we form a lightweight multimodal component to capture cross-modal collaboration while preserving band structure; a residual, task-adaptive gate then aggregates bands into the final representation. To control redundancy and improve generalization, we regularize with a global frequency-domain IB term that allocates capacity across bands (Wiener-style shrinkage of informative directions with shut-off of weak bands). We further introduce a cross-modal spectral consistency loss that aligns modalities. The model is trained end-to-end with a standard objective plus the IB-inspired and consistency terms. In summary, the main contributions of this work are as follows: • We establish a spectral information-theoretic basis. Under an orthogonal or tight-frame transform that approximately block-diagonalizes bands, the Gaussian IB decouples, supporting a frequency-first separate–then–fuse design. • We instantiate this foundation in FITMM, with modality-wise spectral decomposition, within-band multimodal interaction, residual task-adaptive gating, and a global frequency-domain IB term for capacity allocation. • We conduct extensive experiments on three real-world datasets with standard protocols, demonstrating that FITMM’s theoretically grounded design leads to significant performance gains over advanced baselines. 2. RELATED WORK 2.1. Multimodal Recommendation The increasing availability of multimodal data in platforms such as e-commerce and short video services has notably expanded the scope of recommender systems. To move beyond the limitations of traditional collaborative filtering methods ( Koren et al. 2009 ; Rendle et al. 2012 ) , recent studies have explored incorporating various content modalities, including images, textual descriptions and audio, to better capture user preferences and address data sparsity ( Xu et al. 2023 ; Yu et al. 2022 ; Han et al. 2022 ; Yang et al. 2023b ) . Early approaches often treated multimodal information as auxiliary input and combined it with high-order feature interactions to improve performance ( Chen et al. 2019 ; Deldjoo et al. 2021 ; He and McAuley 2016 ; Kang et al. 2017 ) . Building on early multimodal fusion strategies, recent research ( Yang et al. 2023c ; Zhou 2023 ; Geng et al. 2023 ) advances toward fine-grained modeling. To further address the challenges of modality fusion and noise control, several advanced methods have been developed. MMIL ( Yang and Yang 2024 ) introduces a multi-intention framework that jointly learns intention prototypes and cross-modal semantics for personalized recommendation. AlignRec ( Liu et al. 2024b ) improves integration by