Species coexistence in the reinforcement learning paradigm

Kaiwen Jiang, Chenyang Zhao, Shengfeng Deng, Weiran Cai, Jiqiang Zhang, Li Chen · 2025

Read Species coexistence in the reinforcement learning paradigm by Kaiwen Jiang, Chenyang Zhao, Shengfeng Deng, Weiran Cai, Jiqiang Zhang, Li Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17599.txt.

Introduction

Kaiwen Jiang and co-authors use learning agents to study how species can live side by side, in a 2025 paper. Ecology often uses a rock-paper-scissors game as a simple model, where each species beats one and loses to another. In that model, too much movement can wipe out diversity, yet real mobile species often coexist.

Here each individual learns to move with a Q-learning rule instead of moving at a fixed rate. All three species then stayed stable across a wide range of base movement rates. Agents learned to flee predators and stay near prey. Adaptive movers also beat fixed movers when the two met.

Catalog record

CREATORKaiwen Jiang, Chenyang Zhao, Shengfeng Deng, Weiran Cai, Jiqiang Zhang, Li Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-17599

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Species coexistence in the reinforcement learning paradigm Kaiwen Jiang, Chenyang Zhao, Shengfeng Deng, Weiran Cai, Jiqiang Zhang, Li Chen Abstract A central goal in ecology is to understand how biodiversity is maintained. Previous theoretical works have employed the rock-paper-scissors (RPS) game as a toy model, demonstrating that population mobility is crucial in determining the species coexistence. One key prediction is that biodiversity is jeopardized and eventually lost when mobility exceeds a certain value—a conclusion at odds with empirical observations of highly mobile species coexisting in nature. To address this discrepancy, we introduce a reinforcement learning framework and study a spatial RPS model, where individual mobility is adaptively regulated via a Q-learning algorithm rather than held fixed. Our results show that all three species can coexist stably, with extinction probabilities remaining low across a broad range of baseline migration rates. Mechanistic analysis reveals that individuals develop two behavioral tendencies: survival-priority (escaping from predators) and predation-priority (remaining near prey). While species coexistence emerges from the balance of the two tendencies, their imbalance jeopardizes biodiversity. Notably, there is a symmetry-breaking of action preference in a particular state that is responsible for the divergent species densities. Furthermore, when Q-learning species interact with fixed-mobility counterparts, those with adaptive mobility exhibit a significant evolutionary advantage. Our study suggests that reinforcement learning may offer a promising new perspective for uncovering the mechanisms of biodiversity and informing conservation strategies. I introduction Ecological systems are crucial for humans, as they provide materials and energy for our survival Assessment 2005 . Biodiversity is the key property that supports their functional working. Since Darwin first envisioned the “tree of life” Darwin 2004 , understanding the mechanisms underpinning species coexistence has remained a central challenge in ecology Pennisi 2005 . According to the Dasgupta report Government 2021 , biodiversity is declining faster than any time in human history; the current extinction rate of species nowadays is around 100 to 1000 times higher than the baseline value. Decoding how species coexist is a crucial scientific question that may help preserve biodiversity and ultimately promote the sustainability of human civilization. The past several decades have witnessed significant progress in theoretical ecology May and McLean 2007 through the combination of nonlinear dynamics, agent-based modeling, and evolutionary game theory. While the classical population models, such as the Lotka-Volterra model Murray 2013 , can provide a deterministic description of the system in the form of differential equations and can be elegantly solved, they fail to capture the fluctuations and complex interactions Lehman and Tilman 1997 . Agent-based models complement the macroscopical method by allowing for more details and revealing diverse spatiotemporal patterns McLane et al. 2011 . Evolutionary game theory Smith 1982 ; Hofbauer and Sigmund 1998 ; Nowak 2006 contends that the success of one species intrinsically depends on the behavior of others, providing a powerful theoretical framework for population dynamics. Within this context, the rock-paper-scissors (RPS) game has emerged as a canonical model for species diversity Durrett and Levin 1997 ; Durrett and Levin 1998 ; Kerr et al. 2002 ; Czárán et al. 2002 ; May and Leonard 1975 ; Johnson and Seinen 2002 ; Reichenbach et al. 2006 ; Szabó and Fáth 2007 ; Szolnoki et al. 2014 ; Zhou 2016 , where rock is wrapped by paper, paper is cut by scissors, and scissors are crushed by rock. This nonhierarchical, cyclic competition structure captured in RPS game is widely observed in nature, such as lizard populations Sinervo and Lively 1996 , strains of yeast Paquin and Adams 1983 , reef invertebrates Jackson and Buss 1975 , among others Czárán et al. 2002 . Intuition suggests diversity should persist in such systems: an endless pursuit where each species dominates one competitor yet is dominated by another, creating a cyclic hierarchy of advantage. However, this is not necessarily the case when individuals are located in a spatial domain, where they move constantly. In the seminal work by RMF Reichenbach et al. 2007 , they incorporate mobility into a spatially extended RPS model (the RMF model), and they reveal that the three species coexist in the form of entangled spiral waves for low mobility, but extinction occurs when their mobilities exceed a critical value. This prediction is, however, inconsistent with reality, as there are many examples where species with high mobility in nature coexist well with each other. Subsequent research proposes some mechanisms aiming to fill this gap by introducing new ingredients, such as intraspecific competition Yang et al. 2010 , viral/infectious transmission Wang et al. 2010 , cross-patch migration Wang et al. 2011 , habitat suitability Park et al. 2013 , among others Huang et al. 2023 ; Lee et al. 2022 ; Menezes et al. 2022 ; Park and Jang 2020 ; Park 2019 . In particular, in Ref. Park et al. 2013 Junpyo et al. introduce an index to characterize the local habitat suitability whereby individuals adjust their migration; they find robust coexistence even in the high-mobility regime. This work captures a basic biological instinct – individuals are likely to move away when the local habitat becomes hostile and exhibit low mobility in favorable surroundings otherwise. While these works correctly grasp the adaptive nature in mobility, their models hinge on handcrafted heuristic rules that fail to capture the adaptive learning processes inherent to living organisms. Recently, reinforcement learning (RL), a fundamentally different paradigm, offers new perspectives on understanding both social and ecological systems. It has been shown that the emergence of cooperation Ding et al. 2023 ; Zheng et al. 2024a , trust Zheng et al. 2024b , fairness Zheng et al. 2025 , and resource allocation Zheng et al. 2023 , and some other human behaviors Zhang et al. 2020a ; Zhang et al. 2020b can be well understood with RL. Unlike the mechanical models, where individuals make their moves according to some prescribed rates or probabilities, RL players score different actions for different states determined by their environments. This allows them to adjust their actions adaptively and could make completely different moves in response to their surroundings. A key idea behind RL players is that they aim to maximize the accumulated payoffs rather than the immediate rewards, thereby better adapting to their surroundings. However, the studies applying RL in ecology mostly focused on the predator-prey systems Olsen and Fraczkowski 2015 ; Wang et al. 2019 ; Wang et al. 2020 ; Park et al. 2021 , with particular interests in swarming behaviors Li et al. 2023 and collaborative hunting Tsutsui et al. 2024 . A recent work Si and Ito 2025 based on the prisoner’s dilemma game studies a three-species population with Q-learning, but emphasizes sustaining cooperation. Therefore, can RL paradigm offer new insight into the biodiversity? This would provide a more natural explanation for the gap left by the RMF model, i.e., how species with high mobility coexist? In this work, we propose the RL paradigm to identify the mechanism of species coexistence. Specifically, we employ a Q-learning algorithm on a spatial RPS model; individuals belonging to the same species are guided by a common Q-table. This shared Q-table can be interpreted as the collective wisdom passed from their ancestors. For simplicity, individuals are engaged in random exploration to

Read the complete text