Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures

Clément Hongler ∗, Andrew Emil · 2025

Read Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures by Clément Hongler ∗, Andrew Emil (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06832.txt.

Introduction

Clément Hongler and Andrew Emil turn language models into players of formal games built from prediction scores, in a 2025 paper. A language model is, at heart, a probability measure over text. Asking what it means for the model to know that measure leads past plain text generation. Tasks like summarizing, spotting anomalies, hunting for original text, debating, and reverse prompting all become games scored by cross-entropy.

These Xent games can be single-player or multiplayer and are written as small programs. The authors show the game space grows from a few consistency axioms yet holds a wealth of cases. Families of games can then serve as capability benchmarks for models. To tame the endless scope, they suggest exploring the game space with evolution-like dynamics.

Catalog record

CREATORClément Hongler ∗, Andrew Emil
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE153 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06832

Browse the full library · About the Tower of Babel