Multi-LLM Token Filtering and Routing for Sequential Recommendation Conference: ; ; CCS: Computing methodologies Natural language processing
Wuhan Chen, Min Gao, Xin Xia, Zongwei Wang, Wentao Li, Shane Culpepper · 2026
Read Multi-LLM Token Filtering and Routing for Sequential Recommendation Conference: ; ; CCS: Computing methodologies Natural language processing by Wuhan Chen, Min Gao, Xin Xia, Zongwei Wang, Wentao Li, Shane Culpepper (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18200.txt.
Introduction
Chen, Gao, Xia, Wang, Li, and Culpepper ask whether recommendation can benefit from language-model token embeddings alone, without any text input. Injecting tokens from a single model into sequential recommenders proves unstable, because meanings misalign and coverage stays narrow. Their MLTFR framework filters tokens by user guidance and routes knowledge from several models through a Mixture-of-Experts design. Experiments show it beats state-of-the-art sequential baselines and existing alignment methods.
Catalog record
| CREATOR | Wuhan Chen, Min Gao, Xin Xia, Zongwei Wang, Wentao Li, Shane Culpepper |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47934 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18200
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-LLM Token Filtering and Routing for Sequential Recommendation Conference: ; ; CCS: Computing methodologies Natural language processing Wuhan Chen, Min Gao, Xin Xia, Zongwei Wang, Wentao Li, Shane Culpepper Abstract . Large language models (LLMs) have recently shown promise in recommendation by providing rich semantic knowledge. While most existing approaches rely on external textual corpora to align LLMs with recommender systems, we revisit a more fundamental yet underexplored question: Can recommendation benefit from LLM token embeddings alone without textual input? Through a systematic empirical study, we show that directly injecting token embeddings from a single LLM into sequential recommenders leads to unstable or limited gains, due to semantic misalignment, insufficient task adaptation, and the restricted coverage of individual LLMs. To address these challenges, we propose MLTFR , a M ulti- L LM T oken F iltering and R outing framework for corpus-free sequential recommendation. MLTFR follows an interaction-guided LLM knowledge integration paradigm, where task-relevant token embeddings are selected via user-guided token filtering to suppress noisy and irrelevant vocabulary signals. To overcome the limitations of single-LLM representations, MLTFR integrates multiple LLM token spaces through a Mixture-of-Experts architecture, with a Fisher-weighted semantic consensus expert to balance heterogeneous experts and prevent domination during training. By jointly filtering informative tokens and aggregating complementary semantic knowledge across multiple LLMs, MLTFR enables stable and effective utilization of LLM token embeddings without textual inputs or backbone modification. Extensive experiments demonstrate that MLTFR consistently outperforms state-of-the-art sequential recommendation baselines and existing alignment methods. Our code is available at: https://github.com/ccwwhhh/MLTFR. 1. Introduction Recommender systems aim to learn representations of users and items from historical interactions to infer user interests and predict future behaviors, and have been widely deployed in real-world platforms such as e-commerce and media services ( 38 ; 15 ; 39 ) . Sequential recommendation, in particular, models the temporal dynamics of user behaviors and has achieved strong performance with Transformer-based architectures. Recently, the rapid progress of large language models (LLMs) ( 26 ) has extended their influence beyond natural language processing. To leverage the rich semantic knowledge of LLMs for recommendation, a growing body of work integrates LLMs into downstream recommender systems, where LLMs serve as standalone recommenders or as semantic encoders that enrich item representations using textual attributes such as titles, descriptions, or reviews ( 44 ; 6 ; 40 ) . While effective, these approaches rely heavily on the availability of high-quality textual corpora, which may be unavailable, incomplete, or costly to maintain in many practical settings. Figure 1 . Illustration and empirical evidence of relevant token selection. The upper panel illustrates filtering noisy LLM token embeddings into task-relevant tokens. The lower panel compares different token embeddings on the SASRec backbone across two datasets. The dashed line represents the baseline performance without LLM integration. TE and RE denote token embeddings directly extracted from LLMs and randomly initialized embeddings, respectively. Beyond external text, LLMs inherently encode rich semantic knowledge within their parameters, particularly in their token embedding spaces, which capture entity-level semantics learned during large-scale pretraining ( 9 ; 49 ) . This observation suggests that LLM token embeddings may provide transferable semantic structure that complements ID-based item representations at low cost. Intuitively, since a user’s interaction history can be viewed as a sequence of item entities, token embeddings offer a natural medium for injecting semantic signals into sequential recommendation models, even in the absence of textual corpora. This raises a natural question: Can LLM token embeddings alone improve sequential recommendation performance? To investigate whether LLM token embeddings alone can improve sequential recommendation ( 34 ; 10 ; 15 ) , we study a corpus-free setting, where no item-level textual information is available during training or inference. As a preliminary attempt, we directly integrate token embeddings extracted from pretrained LLMs into a sequential recommender by projecting them to match the input embedding dimensionality and adding them to sequence embeddings via element-wise addition (denoted as LLM-TE ). As a control, we replace these token embeddings with randomly initialized embeddings of the same shape ( LLM-RE ). As shown in Fig. 1 , directly injecting LLM token embeddings yields better performance than random embeddings, indicating that LLM token embeddings indeed encode transferable semantic information. However, the resulting gains are limited and unstable, and consistently underperform the original backbone. This observation suggests that naïvely incorporating LLM token embeddings introduces substantial noise and semantic misalignment, motivating the need for task-relevant token selection. We identify two fundamental challenges underlying this phenomenon. First, LLM vocabularies are large and generic, whereas user interests in sequential recommendation are sparse and task-specific; consequently, only a small fraction of token embeddings is relevant to a given user’s interaction history, while the majority behaves as noise. Second, relying on a single LLM provides a limited semantic perspective, leading to coverage gaps and unstable training dynamics when token embeddings are directly injected. Motivated by these observations, we propose M ulti- L LM T oken F iltering and R outing ( MLTFR ), a plug-and-play framework for corpus-free sequential recommendation. MLTFR follows an interaction-guided LLM knowledge integration paradigm, treating LLM token embeddings as lightweight semantic resources rather than full reasoning modules. To address the first challenge, MLTFR introduces a user-guided token filtering mechanism that selects a compact set of task-relevant tokens by measuring their relevance to user interaction representations, thereby suppressing noisy and irrelevant vocabulary signals. To address the second challenge, MLTFR integrates token embeddings from multiple LLMs through a Mixture-of-Experts (MoE) architecture ( 45 ; 25 ; 7 ) , enabling complementary semantic knowledge to be leveraged across models. A key difficulty in multi-LLM integration is expert domination, where a single LLM disproportionately influences training and degrades robustness. To mitigate this issue, MLTFR introduces a semantic consensus expert based on Fisher-weighted aggregation ( 24 ; 3 ) , which fuses shared token-level signals across LLMs while down-weighting conflicting cues ( 18 ) . Combined with user-guided token filtering, this dual-module design enables stable and effective utilization of LLM token embeddings without requiring textual corpora or modifying backbone architectures. Our contributions can be summarized as: • We propose a corpus-free paradigm, Interaction-Guided LLM Knowledge Integration , which leverages LLM token embeddings as lightweight semantic resources for sequential recommendation without relying on textual inputs. • We introduce a user-guided token filtering strategy that selects task-relevant LLM tokens aligned with user interests, effectively suppressing noisy vocabulary signals and improving recommendation relevance. • We develop a shared expert MoE routing mechanism to integrate multiple LLMs, forming a semantic consensus that mitigates cross-LLM conflicts and enables robust and adaptable performance. 2. Methodology We study the standard next-item sequential recommendation task: given a user’s chronological interaction sequence X_{u}=\{i_{1}^{u},i_{2}^{u},\ldots,i_{k}^{u}\} , our goal is to predict the next item i_{k+1}^{u} . Figure 2 summarizes MLTFR as a plug-and-play module inserted on top of an arbitrary ID-based sequential recommender. Concretely, we first embed the input sequence into \mathbf{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{k}\}\in\mathbb{R}^{k\times d_{\text{emb}}} . MLTFR then enriches \mathbf{E} with semantics extracted from multiple LLM vocabularies: For each expert m , MLTFR (i) filters a task-relevant subset of tokens from the corresponding LLM vocabulary, and (ii) derives a knowledge matrix \mathbf{W}_{\text{expert}_{m}}\in\mathbb{R}^{k\times d_{\text{emb}}} from the selected token embeddings via cross-attention with \mathbf{E} . A gating network subsequently performs item-level soft routing to