Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjong
Jim O’Connor, Derin Gezgin, Gary B. Parker · 2025
Read Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjong by Jim O’Connor, Derin Gezgin, Gary B. Parker (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07522.txt.
Introduction
Jim O'Connor and co-authors train a game AI for Sparrow Mahjong by evolution in a 2025 paper. The game mixes luck, hidden data, and long plans. Gradient-based learning is not the only path to strong play. Their agent, Evo-Sparrow, pairs a memory network with an evolution strategy.
The network reads board states and scores each choice. An evolution method tunes its weights by trial populations, not by backpropagation. Over many simulated games the agent beat random and rule-based rivals. It matched a strong reinforcement learning baseline. The hybrid route costs less compute than classic training. Lessons may carry to other stochastic games and adaptive decision tasks.
Catalog record
| CREATOR | Jim O’Connor, Derin Gezgin, Gary B. Parker |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07522
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjong Jim O’Connor, Derin Gezgin, Gary B. Parker Abstract We present Evo-Sparrow, a deep learning-based agent for AI decision-making in Sparrow Mahjong, trained by optimizing Long Short-Term Memory (LSTM) networks using Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Our model evaluates board states and optimizes decision policies in a non-deterministic, partially observable game environment. Empirical analysis conducted over a significant number of simulations demonstrates that our model outperforms both random and rule-based agents, and achieves performance comparable to a Proximal Policy Optimization (PPO) baseline, indicating strong strategic play and robust policy quality. By combining deep learning with evolutionary optimization, our approach provides a computationally effective alternative to traditional reinforcement learning and gradient-based optimization methods. This research contributes to the broader field of AI game playing, demonstrating the viability of hybrid learning strategies for complex stochastic games. These findings also offer potential applications in adaptive decision-making and strategic AI development beyond Sparrow Mahjong. 1 Introduction Game playing artificial intelligence (AI) has long served as a benchmark and testbed for the continued advancement of the field of AI. From early rule-based systems to modern deep reinforcement learning approaches, AI agents have demonstrated superhuman performance in various domains, including board games such as chess ( Campbell, Hoane, and Hsu 2002 ) , checkers ( Samuel 1959 ) , and Go ( Silver et al. 2017b ) . These advances have been driven by a combination of search-based techniques, such as minimax with alpha-beta pruning ( Knuth and Moore 1975 ) , Monte Carlo Tree Search (MCTS) ( Coulom 2006 ) , and deep learning-based policy optimization, as exemplified by AlphaGo, AlphaZero, and MuZero ( Silver et al. 2016 ; Silver et al. 2017a ; Schrittwieser et al. 2019 ) . However, while deterministic games with perfect information have been well studied, non-deterministic games with hidden information and stochastic elements continue to pose significant challenges ( Brown, Sandholm, and Amos 2018 ) . The evolution of AI for game playing has followed a trajectory of increasing complexity, beginning with early symbolic AI and rule-based systems that relied on handcrafted strategies. Early successes in this domain were driven by algorithms such as minimax, which effectively modeled decision-making in fully observable, deterministic games like chess and checkers ( Shannon 1950 ) . The introduction of heuristic evaluations and pruning techniques, such as alpha-beta pruning, enabled AI to search deeper into game trees with reduced computational costs ( Knuth and Moore 1975 ) . The next major breakthrough in game playing AI came with probabilistic methods and machine learning-based approaches. Temporal Difference (TD) learning, exemplified by TD-Gammon ( Tesauro 1995 ) , demonstrated the potential of reinforcement learning to develop sophisticated strategies without explicit domain knowledge. This system paved the way for methods that combined reinforcement learning with tree search, such as MCTS, which was instrumental in advancing AI performance in games like Go ( Gelly et al. 2012 ) . MCTS’s success stemmed mainly from its ability to balance exploration and exploitation efficiently, making it well-suited for large decision spaces where exhaustive search was computationally infeasible. Deep learning further revolutionized AI game playing by enabling agents to learn directly from structured data. The introduction of deep reinforcement learning, particularly with advancements like Deep Q-Networks (DQN) ( Mnih et al. 2013 ) , showed that neural networks could approximate value functions and learn policies for complex environments. The combination of deep learning with self-play and policy optimization, as demonstrated by AlphaGo and its successors AlphaZero and MuZero, achieved superhuman performance in Go, chess, and shogi. These advancements highlighted the power of combining deep neural networks with reinforcement learning and search techniques, setting new standards for AI in strategic decision-making. Mahjong, a traditional tile-based game originating in China, represents a complex domain for AI research due to its vast state space, elements of randomness, and hidden information. Variants of Mahjong have been studied within the AI community, with researchers exploring rule-based heuristics, reinforcement learning, and search-based strategies to develop competent agents ( Li et al. 2020 ) . Among these variants, a simplified three-player version of the game called Sparrow Mahjong ( Koyamada et al. 2023 ) , shown in Figure 1 , presents an intriguing testbed for AI due to its strategic depth, partial observability, and non-deterministic nature. Unlike classic four-player Mahjong, Sparrow Mahjong reduces the number of tiles per player, accelerating the decision-making process while retaining core strategic elements. Figure 1: An example game state in Sparrow Mahjong, visualized in the PGX environment. Traditional approaches to game playing AI, such as minimax search, have limited applicability in non-deterministic, partially observable environments. Probabilistic reasoning and policy learning techniques have been employed to address these challenges in games such as poker ( Brown and Sandholm 2019 ) and Hanabi ( Bard et al. 2020 ) , where bluffing and information inference play crucial roles. In contrast to these traditional and probabilistic methods, recent advancements in deep learning and Long Short-Term Memory (LSTM) networks ( Hochreiter and Schmidhuber 1997 ) have enabled AI systems to process sequential dependencies in decision-making, making these systems well-suited for games that require long-term strategic planning. In this work, we present Evo-Sparrow, a novel approach to training an AI agent for Sparrow Mahjong by optimizing the weights of an LSTM model using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) ( Ostermeier, Gawelczyk, and Hansen 1994 ) . Evolutionary strategies such as CMA-ES have demonstrated effectiveness in optimizing high-dimensional neural network policies without requiring explicit gradient computation, making them a compelling alternative to traditional reinforcement learning or gradient-based optimization techniques. Our approach leverages a simple rule-based baseline to provide a structured evaluation metric, allowing us to quantify performance improvements and evaluate the efficacy of the model. 2 Related Work Mahjong has been a challenging benchmark for artificial intelligence research, due to its high-dimensional state space, stochastic nature, and partial observability ( Tang, Chen, and Wu 2025 ) . According to Chen, Tang, and Wu 2022 , the state space complexity of Taiwanese Mahjong is estimated to be around 4.3\times 10^{185} , which exceeds the complexity of GO ( 10^{172} ) and chess ( 10^{46} ) ( van den Herik, Uiterwijk, and van Rijswijck 2002 ) . Moreover both Li et al. 2024 and Lu, Li, and Li 2023 states that the information size of mahjong is 10^{48} . Early attempts to develop a game-playing agent for Mahjong have primarily focused on rule-based systems and statistical modeling ( Lu, Li, and Li 2023 ) . Monte Carlo simulations and probabilistic opponent modeling have been used to predict the state of opponents, winning tiles, game scores, and upcoming moves ( Mizukami and Tsuruoka 2015 ) . While these approaches were effective in mimicking human-like strategies, they require pre-set features and struggle in generalization. One notable example of these techniques is SIMCAT , which utilizes a regular Monte Carlo approach with heuristic pruning for a Taiwanese variant of Mahjong. SIMCAT achieved the top placement in the Computer Olympiad 2020 Mahjong Tournament ( Chen, Tang, and Wu 2022 ) . More recent efforts in Mahjong AI have utilized deep learning. For example, a convolutional neural network (CNN) based system trained on game