SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?

Jianzhu Yao, Kevin Wang, Ryan Hsieh, Haisu Zhou, Tianqing Zou, Zerui Cheng, Zhangyang Wang, Pramod Viswanath · 2025

Read SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially? by Jianzhu Yao, Kevin Wang, Ryan Hsieh, Haisu Zhou, Tianqing Zou, Zerui Cheng, Zhangyang Wang, Pramod Viswanath (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12349.txt.

Description

Jianzhu Yao and co-authors published a computing paper in 2025 benchmarking how well language models plan strategically and reason socially. Social strategy is harder than solo puzzles, since success depends on reading partners and rivals. Their benchmark, SPIN-Bench, mixes classic planning tasks, competitive board games, cooperative card games, and negotiation scenes in one frame. An arena lets agents play these settings out.

The team varies action spaces, state complexity, and agent counts to stress different skills. Today's models handle fact recall and short plans decently. They stumble on deep multi-step reasoning over big state spaces and on coordination under uncertainty. The authors offer the bench to spur work on multi-agent planning and human-AI teaming.

Catalog record

CREATORJianzhu Yao, Kevin Wang, Ryan Hsieh, Haisu Zhou, Tianqing Zou, Zerui Cheng, Zhangyang Wang, Pramod Viswanath
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12349

Browse the full library · About the Tower of Babel