Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models

Daren Zhong, Dingcheng Huang, Clayton Greenberg · 2025

Read Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models by Daren Zhong, Dingcheng Huang, Clayton Greenberg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01880.txt.

Introduction

Chess engines are built to find the best move, but people rarely play the best move. A 2025 paper by Daren Zhong, Dingcheng Huang, and Clayton Greenberg takes the human side instead, trying to predict which move a person will actually choose, given their skill level.

Their tool is a family of n-gram language models, one trained for each skill group, that learn the patterns in how players at that level continue a game. Because beginners and experts make different kinds of choices, separate models capture habits a single model would blur. The work offers a window into human decision-making, not just machine strength.

Catalog record

CREATORDaren Zhong, Dingcheng Huang, Clayton Greenberg
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01880

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models Daren Zhong, Dingcheng Huang, Clayton Greenberg Abstract Chess, a deterministic game with perfect information, has long served as a benchmark for studying strategic decision-making and artificial intelligence. Traditional chess engines or tools for analysis primarily focus on calculating optimal moves, often neglecting the variability inherent in human chess playing, particularly across different skill levels. To overcome this limitation, we propose a novel and computationally efficient move prediction framework that approaches chess move prediction as a behavioral analysis task. The framework employs n-gram language models to capture move patterns characteristic of specific player skill levels. By dividing players into seven distinct skill groups, from novice to expert, we trained separate models using data from the open-source chess platform Lichess. The framework dynamically selects the most suitable model for prediction tasks and generates player moves based on preceding sequences. Evaluation on real-world game data demonstrates that the model selector module within the framework can classify skill levels with an accuracy of up to 31.7% when utilizing early game information (16 half-moves). The move prediction framework also shows substantial accuracy improvements, with our Selector Assisted Accuracy being up to 39.1% more accurate than our benchmark accuracy. The computational efficiency of the framework further enhances its suitability for real-time chess analysis. 1 Introduction Chess, a deterministic game with perfect information, serves as a well-established benchmark for studying strategic decision-making and artificial intelligence. Its inherent complexity, with approximately 10^{120} possible game variations, makes it an ideal testbed for computational reasoning 5 . Significant advancements in chess engines have evolved from Alan Turing’s pioneering 1948 Turochamp to modern systems such as Stockfish and AlphaZero 7 ; 6 . These engines primarily focus on calculating optimal moves using deep search algorithms, reinforcement learning, and position evaluation functions, excelling in predicting the most advantageous continuations from given board states 7 ; 4 . Most existing approaches to chess analysis focus on optimal move generation, which neglects the inherent variability and uncertainty characteristic of human decision-making 1 . While determining optimal moves is useful to build the best chess engine, it does not adequately capture behavioral patterns. Conventional analysis often relies on evaluating human moves against computer-generated optimal moves, focusing on strategic correctness rather than the players’ tendencies. In addition, chess has been studied through the lens of natural language. Previous research has fine-tuned a GPT-2 model to learn chess using PGN notation and demonstrated that it can play and utilize high-level early-game strategies well 4 . However, this research, too, focuses primarily on strategic correctness. To address this gap, the proposed approach models chess move prediction as a language analysis problem rather than an optimal move calculation task. The framework utilizes n-gram language models, typically employed in natural language processing, to represent chess moves as sequences of words or tokens. Conceptualizing moves as linguistic elements enables the analysis of how move patterns differ across skill levels, revealing player tendencies that are not necessarily aligned with optimal play. The primary objective is to develop a move prediction framework that reflects player behavior across a range of skill levels, from novices to experts. Players were categorized based on their ratings, and the model analyzed move sequences to identify their skill level. Rather than selecting the move with the highest confidence from all models, the framework applied the model most consistent with the player’s skill level, resulting in improved prediction accuracy and computational efficiency. To validate the framework, we used data from the open-source chess platform Lichess. The dataset, containing games categorized by player rating, was preprocessed to eliminate redundant and inconsistent elements, ensuring reliable input for model training. The evaluation compared the framework’s performance against a designed benchmark, using both Top-1 and Top-3 move prediction metrics. The results demonstrate that selectively applying skill-specific models improves accuracy, especially in scenarios where player moves exhibit inherent uncertainty. The proposed framework offers a novel perspective on chess analysis, emphasizing play choices rather than solely focusing on optimal strategies. By identifying player skill levels and employing efficient move prediction techniques, the framework provides insights into how strategic choices evolve with increasing skill, offering a robust foundation for future research in human-centric chess analysis. 2 Dataset and Data Processing 2.1 Dataset We utilized data from the open-source chess platform lichess.org ’s publicly available archive at database.lichess.org , which contains all standard-rated games played monthly since January 2013. The original dataset was distributed in the PGN (Portable Game Notation) format and compressed using zStandard. PGN files encode each game as a combination of structured metadata (such as player ratings, time controls, and event information) and the full sequence of moves. The sequence of moves also contains elements such as move numbers, computer-generated evaluations (e.g., [%eval -0.29]), symbolic annotations (e.g., "!" or "?"), and side variations. While these elements are useful for human analysis, they introduce inconsistencies that complicate language model training and are rarely needed in practical applications. Games were placed in skill groups based on the average of each game’s "WhiteRating" and "BlackRating" headers. We used the following bins: Level 1 (L1): <=1000 Level 2 (L2): 1000–1400 Level 3 (L3): 1400–1600 Level 4 (L4): 1600–1800 Level 5 (L5): 1800–2000 Level 6 (L6): 2000–2250 Level 7 (L7): >=2250 This grouping scheme was informed by the empirical distribution of Lichess ratings and common distinctions in player skill levels. The rating distribution used for stratification is shown in Fig. Figure 1: The distribution considered for analysis To address hardware constraints when training language models, we took a random 10% of each level’s total games from July 2024 as training data. We also took 1000 games from August 2024 as testing data. 2.2 Processing Pre-processing was a critical step in transforming raw PGN files into a form suitable for model training and inference. In particular, our goal was to standardize formatting to improve computational efficiency. Computational efficiency and an efficient input format improve the model’s usability in real-world applications. Furthermore, because the models we used were n-gram, they are limited to a small number of words as history. Making the input format more efficient likely improved model accuracy as the model could access more relevant history. After decompressing the data using a stream reader, we initially considered parsing the PGN files using the python-chess library. However, we ultimately opted to treat the PGNs as plain text, which significantly accelerated processing by bypassing legality checks and other parsing overhead. Metric Purpose Perplexity Measures how well a model predicted a sequence; Surprisal Measures how well a model predicted a word; 16 Half-Move Accuracy Assess performance of early game inference model selector 100 Half-Move Accuracy Assess performance of full game inference model selector Accuracy A measure for how well the classifier worked Average Error Another measure for determining how well the classifier worked Table 1: Metric and description of its purpose We began by extracting and averaging the "WhiteRating" and "BlackRating" headers to assign each game to its corresponding skill group. Subsequently, we removed all metadata headers, side variations, and any elements not part of the main line of play. The games at this point still included move numbers and evaluations, as illustrated in the

Read the complete text