Manifold Trajectories in Next-Token Prediction: From Replicator Dynamics to Softmax Equilibrium
Christopher R. Lee-Jenkins · 2025
Read Manifold Trajectories in Next-Token Prediction: From Replicator Dynamics to Softmax Equilibrium by Christopher R. Lee-Jenkins (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21186.txt.
Description
Lee-Jenkins re-derives how a language model picks its next word, and finds a familiar dynamic hiding inside. The standard picture is simple bookkeeping: score every candidate word, soften the scores, and normalize into probabilities. Lee-Jenkins shows that step can be read as a constrained optimization on the probability simplex, the space of all possible word distributions.
Done step by step, the update matches a classic multiplicative-weights rule. Smoothed into continuous time, it becomes the replicator flow known from evolutionary theory. Along that path, the distribution glides smoothly to the usual softmax answer. Temperature, he proves, merely rescales time along the same path, and top-k or nucleus sampling just fence the flow onto one face. He also sketches how path-shaped tweaks to scores might mimic hallucination-like loops.
Catalog record
| CREATOR | Christopher R. Lee-Jenkins |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-21186