Next-Token Prediction and Regret Minimization

Mehryar Mohri, Clayton Sanford, Jon Schneider, Kiran Vodrahalli, Yifan Wu · 2026

Read Next-Token Prediction and Regret Minimization by Mehryar Mohri, Clayton Sanford, Jon Schneider, Kiran Vodrahalli, Yifan Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28499.txt.

Description

Mohri and four co-authors ask when a next-token predictor can drive good choices against an adversary. The induced player best-responds to the model's forecast of the foe's moves. With an unbounded context window, every action distribution sits very close to one that yields low regret, so cheap fixes always work. With a bounded window, as in real transformers, some distributions stay far from all low-regret ones. They also show transformer layers can carry out the fix.

Catalog record

CREATORMehryar Mohri, Clayton Sanford, Jon Schneider, Kiran Vodrahalli, Yifan Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94466
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28499

Browse the full library · About the Tower of Babel