Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs

Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen · 2026

Read Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs by Wenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27255.txt.

Description

Tan and co-authors speed up decoding by pairing input compression with multi-token prediction. Their design treats a latent compressor and a prediction head as mirror images. The compressor packs two input tokens into one latent form, while the head unfolds one hidden state into an extra output token. A confidence head accepts or rejects draft tokens, removing the verifier pass, and it trains alongside on-policy distillation at negligible cost. Tests show gains of up to about 7 points on pass-at-4 scores and more than twice faster latency.

Catalog record

CREATORWenhui Tan, Minghao Li, Xiaoqian Ma, Siqi Fan, Xiusheng Huang, Liujie Zhang, Ruihua Song, Weihang Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50581
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27255

Browse the full library · About the Tower of Babel