Variable-Length Semantic IDs for Recommender Systems CCS: Information systems Personalization CCS: Information systems Recommender systems CCS: Computing methodologies Neural networks

Kirill Khrylchenko · 2026

Read Variable-Length Semantic IDs for Recommender Systems CCS: Information systems Personalization CCS: Information systems Recommender systems CCS: Computing methodologies Neural networks by Kirill Khrylchenko (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16375.txt.

Description

Kirill Khrylchenko, writing alone, tackles a scaling problem for generative recommender systems. Such models treat user behavior as event sequences and fold large language models into recommendation pipelines. But item spaces are extremely large, which makes training hard and leaves a gap between natural language and item identifiers. Semantic identifiers, which represent items as strings of low-cardinality tokens, offer an effective answer.

Existing methods give every item a fixed-length code, wasting effort, fitting natural language poorly, and ignoring the skew of real catalogs. There, popular and rare long-tail items need very different amounts of information. Emergent communication research offers a parallel: agents there develop discrete protocols with variable-length messages, giving frequent concepts shorter descriptions. He bridges the two fields with variable-length semantic IDs, learned by a discrete variational autoencoder with Gumbel-Softmax reparameterization, avoiding REINFORCE instability and fixed-length limits.

Catalog record

CREATORKirill Khrylchenko
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16375

Browse the full library · About the Tower of Babel