X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation

Guy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach · 2025

Read X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation by Guy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20859.txt.

Introduction

Guy Hadad and co-authors present X-Cross in a 2025 paper, a model that recommends products in new domains by blending domain-specific language models. Shops launch new catalogs daily, and recommenders must adapt without full retraining.

Each source model is fine-tuned with small low-rank adapters. For a given prompt, X-Cross refines each model's internal representation layer by layer, mixing knowledge from all the others while adapter activations preserve each domain's flavor. On Amazon sequential recommendation data, quality matched a fully fine-tuned rival while using only a quarter of the extra parameters. Moving from toys to tools, electronics, or sports, it needed far less fine-tuning data and beat other cross-domain baselines clearly.

Catalog record

CREATORGuy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20859

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation Guy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach Abstract . As new products are emerging daily, recommendation systems are required to quickly adapt to possible new domains without needing extensive retraining. This work presents “X-Cross” – a novel cross-domain sequential-recommendation model that recommends products in new domains by integrating several domain-specific language models; each model is fine-tuned with low-rank adapters (LoRA). Given a recommendation prompt, operating layer by layer, X-Cross dynamically refines the representation of each source language model by integrating knowledge from all other models. These refined representations are propagated from one layer to the next, leveraging the activations from each domain adapter to ensure domain-specific nuances are preserved while enabling adaptability across domains. Using Amazon datasets for sequential recommendation, X-Cross achieves performance comparable to a model that is fine-tuned with LoRA, while using only 25% of the additional parameters. In cross-domain tasks, such as adapting from Toys domain to Tools , Electronics or Sports , X-Cross demonstrates robust performance, while requiring about 50%-75% less fine-tuning data than LoRA to make fine-tuning effective. Furthermore, X-Cross achieves significant improvement in accuracy over alternative cross-domain baselines. Overall, X-Cross enables scalable and adaptive cross-domain recommendations, reducing computational overhead and providing an efficient solution for data-constrained environments. Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730117 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Recommender systems CCS: Information systems Cross-domain recommendation CCS: Computing methodologies Natural language processing CCS: Computing methodologies Language models CCS: Computing methodologies Transfer learning 1. Introduction The increasing variety of products and services available across online platforms, along with the rapid emergence of new domains, underscores the growing need for sequential recommendation models with fast and efficient cross-domain adaptation. Traditional sequential recommendation models, often tailored to specific content areas, struggle to generalize effectively as new domains emerge at an unprecedented pace. As a motivating example, online marketplaces frequently introduce new and particular product categories, such as “vintage collectibles”, “sustainable fashion”, “personalized DIY kits” or “specialized home automation devices”. These emerging sub-domains challenge conventional models trained on broader categories like “consumer electronics” or “home goods”, since it requires nuanced understanding to capture users’ preferences in such niche areas. As large language models (LLMs) continue to develop in scope and expressive power, they offer exciting potential for cross-domain applications by leveraging their vast world knowledge to quickly adapt to new domains and tasks ( Zhao et al., 2023 ) . This capacity to operate across numerous and increasingly specific domains positions LLMs as powerful tools for recommendation tasks. However, effective cross-domain recommendation with LLMs presents significant challenges. The large size and complexity of state-of-the-art language models make them resource-intensive to train even with parameter-efficient fine-tuning (PEFT) techniques. While PEFT, such as Low-Rank Adaptation (LoRA) ( Hu et al., 2022 ) , has reduced the need for extensive retraining, the parameter requirements for adapting large models across multiple domains remains high, often surpassing the computational budgets of many real-world applications. A key challenge is ensuring that cross-domain recommendation tasks capture domain-specific granularity while remaining adaptable across domains. Current methods ( Buehler and Buehler, 2024 ; Xu et al., 2024 ) rely heavily on the activations of pre-trained weights, which may lack the capacity to generate sufficiently distinctive representations for relatively similar domains. Consequently, these approaches might not adequately capture the unique nuances of each domain. The need for adaptability in cross-domain recommendation is intensified by the data demands of LLMs. These models typically require large, domain-specific datasets to achieve optimal performance, which is not always available – especially in newly emerging domains where data is scarce or when domain-nuanced recommendations are required. To the best of our knowledge, such an approach, which introduces finer-grained adaptability across the model, has not yet been explored in existing research. The goal of our work is, therefore, to leverage language models that were trained with user-item interaction histories observed in other (source) domains for sequential recommendation in a new (target) domain. Hence, we wish to learn to transfer knowledge that was acquired in source domains to the new target domain; this, while using as minimum as possible observed data from the target domain for training. Moreover, we do not assume knowledge sharing of users or items between domains. Trying to address the aforementioned challenges, we introduce the “X-Cross” model – a novel dynamic integration model for cross-domain adaptability that learns to transfer knowledge from several source domain language-models to a new target domain. Given a set of two or more LoRA fine-tuned source domain language models, for each (recommendation prompt) input, X-Cross operates layer-by-layer and computes weights, integrating the strengths of multiple domain-specific models into refined representations. These refined representations are propagated from one layer to another, ensuring domain-specific nuances are preserved while enabling cross-domain adaptability. By leveraging activations from each source domain (LoRA) adapter, the dynamic integration mechanism creates representations that are both granular and adaptable, offering a practical solution. Such a layer-wise dynamic integration eliminates the need to retrain or modify the original source domain LoRA adapters, allowing our model to achieve performance comparable to newly trained LoRA adapters using just 25% of the parameters required by LoRA. Moreover, X-Cross achieves a competitive and sometime even better recommendation quality to a model that is fine-tuned with LoRA while using 50-75% less training data. We further demonstrate that X-Cross performs better than other alternative language-model based cross-domain recommenders, including alternatives that utilize state-of-the-art mixture-of-LoRA methods. Overall, our work contributes to developing a parameter-efficient, data-friendly, and adaptable solution for cross-domain sequential recommendation, offering a scalable pathway for recommendation systems in data-limited and rapidly evolving domains. 2. Related Work We categorize previous related works into three areas of research: language models for sequential recommendation, cross-domain recommendation, and methods employing mixture-of-LoRAs for efficient model fine-tuning. Below, we briefly review these works and highlight their limitations in comparison to our approach. 2.1. Language Models for Sequential-Recommendation The sequential recommendation task has been extensively studied ( Boka et al., 2024 ) , with many previous works commonly model user-item interaction sequences using Transformer-models; having sequence items represented either by their identities (IDs) (e.g., SASRec ( Kang and McAuley, 2018 ) , BERT4Rec ( Sun et al., 2019 ) , SSE-PT ( Wu et al., 2020 ) ) or by their attributes (e.g., FDSA ( Zhang et al., 2019 ) , Trans2D ( Singer et al., 2022 ) ). The rise of large language models (LLMs) in recent years has revolutionized natural language processing (NLP), and specifically their applications to recommendation systems ( Liu et al., 2023 ; Wu et al., 2024 ; Lin et al., 2024 ) (LLM4Rec). For sequential recommendation tasks, language models have been utilized so far in two primary ways. Firstly, language models have been utilized for feature encoding and augmentation, providing rich context for downstream sequential recommendation pipelines ( Lin et al., 2024 ) . Secondly, previous works have utilized the in-context learning and instruction following capabilities of LLMs to provide recommendations

Read the complete text