Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs
Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen · 2026
Read Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs by Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27255.txt.
Description
Tan and co-authors speed up decoding by pairing input compression with multi-token prediction. Their design treats a latent compressor and a prediction head as mirror images. The compressor packs two input tokens into one latent form, while the head unfolds one hidden state into an extra output token. A confidence head accepts or rejects draft tokens, removing the verifier pass, and it trains alongside on-policy distillation at negligible cost. Tests show gains of up to about 7 points on pass-at-4 scores and more than twice faster latency.
Catalog record
| CREATOR | Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50581 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27255