On the Equivalence Between Auto-Regressive Next Token Prediction and Full-Item-Vocabulary Maximum Likelihood Estimation in Generative Recommendation–A Short Note DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Recommender systems
Yusheng Huang, Shuang Yang, Zhaojie Liu, Han Li · 2026
Read On the Equivalence Between Auto-Regressive Next Token Prediction and Full-Item-Vocabulary Maximum Likelihood Estimation in Generative Recommendation–A Short Note DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Recommender systems by Yusheng Huang, Shuang Yang, Zhaojie Liu, Han Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15739.txt.
Description
Huang, Yang, Liu, and Li prove that k-token next-token guessing in generative recommendation matches full-item-vocab top-likelihood guessing exactly. This needs a one-to-one map from items to k-token strings. The match holds for both chain and side-by-side token splits, the two top shop schemes. The proof gives the first firm theory base for the top shop recipe and steers next tuning work.
Catalog record
| CREATOR | Yusheng Huang, Shuang Yang, Zhaojie Liu, Han Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22930 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15739