Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
Clément Hongler ∗, Andrew Emil · 2025
Read Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures by Clément Hongler ∗, Andrew Emil (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06832.txt.
Introduction
Clément Hongler and Andrew Emil turn language models into players of formal games built from prediction scores, in a 2025 paper. A language model is, at heart, a probability measure over text. Asking what it means for the model to know that measure leads past plain text generation. Tasks like summarizing, spotting anomalies, hunting for original text, debating, and reverse prompting all become games scored by cross-entropy.
These Xent games can be single-player or multiplayer and are written as small programs. The authors show the game space grows from a few consistency axioms yet holds a wealth of cases. Families of games can then serve as capability benchmarks for models. To tame the endless scope, they suggest exploring the game space with evolution-like dynamics.
Catalog record
| CREATOR | Clément Hongler ∗, Andrew Emil |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 153 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06832