SLiM-Gym: Reinforcement Learning for Population Genetics
Niko Zuppas · 2025
Read SLiM-Gym: Reinforcement Learning for Population Genetics by Niko Zuppas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16301.txt.
Introduction
Niko Zuppas and co-authors introduce a software package that links learning agents to simulations of evolving populations, in a 2025 paper. The catalog names one author, while the text byline adds a second, so the work is best read as a team product. Population genetics often models groups that change generation by generation under a classic scheme. Teaching a learning agent inside such a world needs a training ground that speaks the agent's language.
The package joins a standard agent interface with a well-known, high-detail evolution simulator. Agents can then act on living populations as the simulation runs, and see the results of their moves. That loop lets researchers build and test agent strategies for steering or probing evolution. The authors frame the tool as a bridge: geneticists gain a ready way to try learning methods, and agent researchers gain a rich, rule-based world to train in.
Catalog record
| CREATOR | Niko Zuppas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 8 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16301
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SLiM-Gym: Reinforcement Learning for Population Genetics
Niko Zuppas
Abstract
Summary
Wright-Fisher evolutionary dynamics provide a mathematical framework for modeling populations over discrete time steps
(
Fisher 1930
)
. Deep reinforcement learning (RL) has proven highly effective in optimizing complex sequential decisions, achieving expert-level performance on tasks such as Poker and Go
(
Heinrich and Silver 2016
;
Silver et al. 2016
)
. However, applying RL to evolutionary problems requires suitable training environments. We present
SLiM-Gym
, a Python package that bridges this gap by connecting the
Gymnasium
RL framework with
SLiM
, a forward-time population genetics simulator, enabling researchers to apply RL methods to study evolutionary processes and generate novel hypotheses.
Statement of Need
Evolutionary trajectories are shaped by the interaction of genetic forces acting sequentially across generations. While theoretical models like Wright-Fisher capture these dynamics with well-defined transition probabilities, unified tools for studying and controlling evolutionary processes through reinforcement learning have been limited. Recent work demonstrated the potential of reinforcement learning for controlling ecological and physiological aspects of evolution
(
Bagley et al. 2025
)
, optimizing breeding programs
(
Younis et al. 2024
)
and addressing emergent drug resistance
(
Engelhardt 2020
)
. However, these implementations often rely on custom simulation tools, potentially limiting accessibility for population geneticists. Reinforcement learning has also been applied to phylogenetic tree space
(
Azouri et al. 2023
)
and ancestral recombination graph construction
(
Raymond et al. 2024
)
, highlighting the broader potential of RL in evolutionary biology.
SLiM
(
Haller and Messer 2023
)
has emerged as a popular forward-time population genetics simulator, with extensive adoption among population geneticists. It excels at modeling complex evolutionary scenarios with individual-level resolution and supports extensive customization through its Eidos scripting language. However, it lacks agent-based optimization or exploration.
Gymnasium
(
Towers et al. 2024
)
, originally released as Gym by OpenAI in 2016
(
Brockman et al. 2016
)
, provides a standardized framework for RL research, while environments like MuJoCo, an advanced physics simulator
(
Todorov et al. 2012
)
, illustrate the value of coupling RL with high-fidelity simulations.
SLiM-Gym
combines
SLiM
’s evolutionary modeling with
Gymnasium
’s RL framework, allowing us to leverage RL to generate testable hypotheses about evolutionary processes and outcomes.
SLiM-Gym
Wrapper
The base wrapper extends
Gymnasium
with modifications for
SLiM
integration. When initializing a new training episode, the wrapper launches
SLiM
as a subprocess that runs a user-defined simulation recipe as an environment an agent can interact with. Communication between
Gymnasium
and
SLiM
occurs through a file-based communication protocol, enabling real-time manipulation of simulation parameters. The wrapper provides abstract methods that environments must implement to handle state processing, action translation, and reward calculation.
Communication Protocol
Figure 1:
Communication protocol between SLiM-Gym and SLiM recipe. Synchronization between SLiM and SLiM-Gym is achieved through two mechanisms: SLiM’s output logging and the signaling file.
The
SLiM-Gym
communication protocol begins by starting the
SLiM
subprocess and initializing a signaling file. During initialization and burn-in generations,
SLiM
operates independently of Python. Once this phase is complete, synchronization between both
SLiM
and
SLiM-Gym
is achieved through two mechanisms:
SLiM
’s output logging and the signaling file. After burn-in,
SLiM
logs generation data and deletes the signaling file, entering a waiting state. The wrapper’s step function monitors for both presence of the log file and absence of the signaling file, proceeding to execute when both conditions are met. The action generated by the learning algorithm is written to a new signaling file, lifting
SLiM
’s waiting condition and allowing the action to be applied to the simulation before continuing. The process is visualized in
Figure 1
.
Reference Environment
Figure 2:
Site frequency spectra across three demographic scenarios. The neutral model exhibits the expected 1/frequency distribution, with the proportion of variants declining smoothly as allele frequency increases. The bottleneck model shows characteristic depletion of rare variants and enrichment of intermediate frequency alleles, reflecting the loss of low frequency variants during population contraction. The growth model shows an excess of rare variants, a signature of recent population expansion when numerous new mutations have not had time to reach higher frequencies. These distinct patterns demonstrate how population size history can shape genetic diversity in populations.
We provide a
SLiM
recipe as a reference environment for modeling a
K
-allele Wright–Fisher process with mutation, formulated as a partially observable Markov decision process (POMDP). While the classical Wright–Fisher model assumes a fixed population size,
SLiM-Gym
relaxes this constraint by allowing the population size
N
to vary over time according to a specified growth or bottleneck model. The agent interacts with the system by dynamically adjusting the mutation rate
\mu
in response to the changing size of
N
. This reference environment explores parameter relationships in evolutionary dynamics. Through reinforcement learning, we aim to determine whether an agent can learn to infer and compensate for unobserved demographic changes by adjusting the mutation rate
\mu
to maintain genetic diversity, discovering the functional relationship between these parameters.
The underlying simulation state at generation
n
includes the allele frequency vector
X_{n}^{N}=(p_{1},\ldots,p_{K})
, where
p_{i}
represents the proportion of derived allele
i
in the population, and a mutation parameter
\mu
. In each generation, the agent selects an action that modifies and sets the next mutation rate
\mu_{n+1}
.
SLiM
then applies evolutionary processes including mutation according to this rate, and draws the resulting allele counts for the next generation from a multinomial distribution with parameters
N_{n+1}
and the post-mutation frequencies.
The agent does not directly observe
X_{n}^{N}
,
N
, or
\mu
. Instead, it observes a site frequency spectrum (SFS—see
Figure 2
for more detail), represented as a fixed-length, 100-element vector where each entry corresponds to a 1% frequency bucket. During an initial burn-in phase, the environment establishes an expected SFS under constant
N
and
\mu
.
The reward function evaluates how well the agent maintains the expected site frequency distribution by calculating the Kullback–Leibler divergence between the observed and expected SFS distributions. Population size
N
is consistently modified by a hidden demographic model and as such the agent must adapt
\mu_{n+1}
based solely on the observed SFS. This provides a theoretically tractable, biologically grounded test case for RL in evolutionary dynamics, with potential extensions to more complex multi-parameter relationships and selective scenarios. For details on custom environments and additional functionality, see the project documentation:
https://github.com/nzupp/SLiM-Gym/
Conclusion
The application of reinforcement learning algorithms to evolutionary scenarios represents an exciting direction for future research. RL offers a novel way to approach open questions in evolutionary biology by allowing agents to develop process-driven hypotheses about evolutionary dynamics.
SLiM-Gym