Large Language Models Enhanced Hyperbolic Space Recommender Systems

Wentao Cheng, Zhida Qin, Zexue Wu, Pengzhan Zhou, Tianyu Huang · 2025

Read Large Language Models Enhanced Hyperbolic Space Recommender Systems by Wentao Cheng, Zhida Qin, Zexue Wu, Pengzhan Zhou, Tianyu Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05694.txt.

Introduction

Wentao Cheng and four co-authors present a 2025 paper on recommender systems that join language models with hyperbolic space. Recommenders guess what a user will like. Flat Euclidean space struggles to hold the tree-like layers in text and meaning data, while curved hyperbolic space fits them well. Blending the two is not simple, though.

Their framework, HyperLLM, works from two sides. It has language models write layered tags for each item, with parent and child links, and learns those beside user-item links. It also draws layered signals from semantic vectors and bridges them to the behavior space. Wide tests showed gains over 40 percent against both hyperbolic and language-based rivals. Training also grew more stable.

Catalog record

CREATORWentao Cheng, Zhida Qin, Zexue Wu, Pengzhan Zhou, Tianyu Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-05694

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models Enhanced Hyperbolic Space Recommender Systems Wentao Cheng, Zhida Qin, Zexue Wu, Pengzhan Zhou, Tianyu Huang Abstract . Large Language Models (LLMs) have attracted significant attention in recommender systems for their excellent world knowledge capabilities. However, existing methods that rely on Euclidean space struggle to capture the rich hierarchical information inherent in textual and semantic data, which is essential for capturing user preferences. The geometric properties of hyperbolic space offer a promising solution to address this issue. Nevertheless, integrating LLMs-based methods with hyperbolic space to effectively extract and incorporate diverse hierarchical information is non-trivial. To this end, we propose a model-agnostic framework, named HyperLLM , which extracts and integrates hierarchical information from both structural and semantic perspectives. Structurally, HyperLLM uses LLMs to generate multi-level classification tags with hierarchical parent-child relationships for each item. Then, tag-item and user-item interactions are jointly learned and aligned through contrastive learning, thereby providing the model with clear hierarchical information. Semantically, HyperLLM introduces a novel meta-optimized strategy to extract hierarchical information from semantic embeddings and bridge the gap between the semantic and collaborative spaces for seamless integration. Extensive experiments show that HyperLLM significantly outperforms recommender systems based on hyperbolic space and LLMs, achieving performance improvements of over 40%. Furthermore, HyperLLM not only improves recommender performance but also enhances training stability, highlighting the critical role of hierarchical information in recommender systems. Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730019 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Recommender systems 1. Introduction Recommender systems have become fundamental to various industries, from e-commerce to content streaming. Traditional recommender techniques ( Karatzoglou and Hidasi, 2017 ; He et al., 2020 ) are effective, but they often face challenges in handling complex relationships between users and items, especially when dealing with large-scale and sparse data. As the demand for personalized recommendations increases, recent advances ( Zhang et al., 2022 ; Xu and Li, 2023 ) in geometric methods provide promising solutions to address these challenges and improve performance. Hyperbolic geometry is a non-Euclidean space that has gained increasing attention for its ability to model hierarchical structures and complex relationships ( Sala et al., 2018 ; Ganea et al., 2018 ; Chami et al., 2019 ) . Unlike the Euclidean space, where spatial capacity grows linearly, hyperbolic space exhibits exponential growth, making it particularly suitable for representing hierarchical data, such as taxonomies and social networks. Research ( Zhang et al., 2021 ; Sun et al., 2021 ; Wang et al., 2021 ) indicates that hierarchical structures exist between users and items in recommender systems. The unique properties of hyperbolic space enable it to effectively model these relationships, thereby better capturing user preferences. In recent years, with the widespread application of Large Language Models (LLMs) ( Kenton and Toutanova, 2019 ; Brown et al., 2020 ) across various fields, their potential to enhance recommender systems has attracted significant attention. A major category of LLMs-based recommender systems ( Ren et al., 2024a ; Liu et al., 2024 ; Wang et al., 2024a ) focuses on utilizing the excellent world knowledge capabilities of LLMs to extract and enrich semantic information from recommender data, addressing challenges such as data sparsity and long-tail effect. However, recent studies ( Chen et al., 2024 ; Yang et al., 2024a ) show that semantic information often contains rich hierarchical structures, and methods relying on Euclidean space may struggle to capture these structures, resulting in performance degradation. The hyperbolic space offers a promising direction to address this issue. However, it is non-trivial to integrate LLMs-based methods with hyperbolic space, which presents challenges arising from two aspects. The first challenge lies in how to extract the diverse hierarchical information hidden within textual and semantic data. The second challenge is the significant gap between the semantic and collaborative information, which makes it difficult to incorporate the extracted hierarchical information into recommender systems. Moreover, this gap is further exacerbated by the differences between hyperbolic and Euclidean spaces. To address these challenges, we extract and integrate hierarchical information from structural and semantic perspectives. From the structural perspective, we focus on well-defined hierarchical relationships in the data. Specifically, we use LLMs to generate multi-level tags with parent-child relationships for each item, representing classifications ranging from broad to specific. These tags are inspired by classification networks ( Horta et al., 2023 ) in real-world taxonomies, which exhibit clear hierarchical structures. To incorporate this information into the hyperbolic model, we construct a tag-item matrix and jointly learn the tag-item and user-item interactions. Subsequently, hyperbolic space contrastive learning ( Qin et al., 2024 ) aligns item representations from both sides, collectively providing structural hierarchical information to the model. Furthermore, we extract semantic hierarchical information to capture more nuanced aspects of the data. Generally, the original textual data describes the objective characteristics of items but rarely reflects subjective user preferences, resulting in a lot of noise unrelated to preferences. Therefore, we use LLMs to summarize preference related information for both users and items. These summaries are converted into semantic embeddings using a text embedding model ( Neelakantan et al., 2022 ) . Then, we propose a novel meta-optimized two-phase training strategy, where each phase corresponds to a distinct training objective. In the first phase, the primary objective is to extract semantic hierarchical information from the semantic embeddings. During this phase, we freeze the parameters of both the semantic embeddings and the hyperbolic model, training only the Mixture of Experts (MoE) model ( Jacobs et al., 1991 ; Shazeer et al., 2017 ; Gale et al., 2023 ) to transform the semantic embeddings effectively. In the second phase, we integrate the extracted hierarchical information into the hyperbolic model for recommendations. For meta-learning methods ( Qin et al., 2023 ; Hao et al., 2024 ) , freezing parameters facilitates the model to learn general knowledge from the target task, which improves generalization on new tasks. To distinguish between these two phases and encourage the model to learn more generalizable knowledge, we design task-specific adaptive margin ranking losses for each phase. Note that the structural and semantic approaches are model-agnostic. In conclusion, we use LLMs to extract structural and semantic hierarchical information and propose a model-agnostic framework, named HyperLLM , which effectively integrates this information into hyperbolic space recommender systems. Consequently, HyperLLM achieves significant performance improvements on multiple hyperbolic baselines, with the highest improvement exceeding 40%. Our contributions can be summarized as follows: \bullet We leverage LLMs to extract both structural and semantic hierarchical information, taking into account the nature of recommender data and the properties of hyperbolic space. This approach enhances the recommender system’s capability to accurately capture hierarchical relationships and user preferences. \bullet We introduce a novel meta-optimized strategy for extracting semantic hierarchical information with hyperbolic model, bridging the gap between semantic and hyperbolic collaborative spaces. This approach also provides a promising direction for semantic fusion in both Euclidean and hyperbolic space recommender systems. \bullet We conduct extensive experiments demonstrating the benefits of structural and semantic hierarchical information. Our proposed HyperLLM framework outperforms existing LLMs-based

Read the complete text