Large Language Model Strategic Reasoning Evaluation through Behavioral Game Theory

Jingru Jia*, Zehua Yuan*, Junhao Pan, Paul E. McNamara, Deming Chen · 2025

Read Large Language Model Strategic Reasoning Evaluation through Behavioral Game Theory by Jingru Jia*, Zehua Yuan*, Junhao Pan, Paul E. McNamara, Deming Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20432.txt.

Description

Jingru Jia and co-authors test, in a 2025 paper whose head titles it as an agentic study through behavioral game theory, how well language models reason when choices depend on rivals. Most past tests scored models only on nearing the game theory ideal. That hides the mechanisms behind their choices.

The authors built a framework that separates raw reasoning skill from context effects. They ran 22 leading models through a battery of games. Three models led most games, yet sheer size did not predict success. Step-by-step prompting helped only models at certain levels. The study gives a clearer view of when a model truly strategizes and when it merely mimics.

Catalog record

CREATORJingru Jia*, Zehua Yuan*, Junhao Pan, Paul E. McNamara, Deming Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20432

Browse the full library · About the Tower of Babel