When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks? Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.372647
Eleni Nisioti, Joachim Winther Pedersen, Erwan Plantec, Milton L. Montero, Sebastian Risi · 2025
Read When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks? Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.372647 by Eleni Nisioti, Joachim Winther Pedersen, Erwan Plantec, Milton L. Montero, Sebastian Risi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22696.txt.
Introduction
Eleni Nisioti and four co-authors ask when evolving neural networks beats reinforcement learning at carrying skills to new tasks, in a 2025 paper. Reinforcement learning dominates high-dimensional control but is brittle when tasks change, and it forgets old skills. Evolution-based training has drawn attention for robustness and escaping local optima, yet its transfer ability was barely studied.
The team built two benchmarks with curricula of rising difficulty. One asks networks to emulate logic circuits with repeating, varying modules. The other extends a physics engine into a robot playground with walls, obstacles, and switchable body shapes. Evolution methods varied among themselves, but they frequently outperformed reinforcement learning baselines on transfer. The authors argue evolution deserves a place as a foundation for adaptable agents, while flagging the challenge of scaling it to messy real-world problems.
Catalog record
| CREATOR | Eleni Nisioti, Joachim Winther Pedersen, Erwan Plantec, Milton L. Montero, Sebastian Risi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22696
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks? Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.3726475 ISBN: 979-8-4007-1465-8/2025/07 Eleni Nisioti, Joachim Winther Pedersen, Erwan Plantec, Milton L. Montero, Sebastian Risi Abstract . The ability to continuously and efficiently transfer skills across tasks is a hallmark of biological intelligence and a long-standing goal in artificial systems. Reinforcement learning (RL), a dominant paradigm for learning in high-dimensional control tasks, is known to suffer from brittleness to task variations and catastrophic forgetting. Neuroevolution (NE) has recently gained attention for its robustness, scalability, and capacity to escape local optima. In this paper, we investigate an understudied dimension of NE: its transfer learning capabilities. To this end, we introduce two benchmarks: a) in stepping gates , neural networks are tasked with emulating logic circuits, with designs that emphasize modular repetition and variation b) ecorobot extends the Brax physics engine with objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. Crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity. Our empirical analysis shows that NE methods vary in their transfer abilities and frequently outperform RL baselines. Our findings support the potential of NE as a foundation for building more adaptable agents and highlight future challenges for scaling NE to complex, real-world problems. † † cc-license: by 1. Introduction Neuroevolution (NE) ( Risi et al. 2025 ; Stanley et al. 2019 ) , is a rich family of diverse algorithms that can serve as a gradient-free optimization technique for artificial neural networks (ANNs) ( Koza and Rice 1992 ; Stanley et al. 2019 ) . In its early years, genetic programming was considered a sample-efficient and scalable alternative to classical reinforcement learning (RL) algorithms ( Stanley and Miikkulainen 2002 ; Koza [n. d.] ; Moriarty and Miikkulainen 1996 ) . Yet, several key innovations in the deep learning era, such as the introduction of novel ANN architectures ( Vaswani et al. 2023 ) , RL objective functions ( Schulman et al. 2017a ) , and optimizers ( Kingma and Ba 2017 ) , have enabled scaling up gradient-based optimization to problems of complexity seemingly unreachable by NE. Recent studies, however, show that there are cases where NE can outperform RL: evolution strategies can provide compute-efficient approximations of gradients ( Salimans et al. 2017 ) , genetic algorithms can avoid local optima ( Such et al. 2018 ) , and Quality-Diversity can lead to improved robustness and generalization ( Chalumeau et al. 2023 ) . Our work contributes to this accumulating evidence by showing that there is another challenge where certain NE methods can offer an advantage: transfer learning. Transfer learning, popularized in the field of machine learning through a need for improved data efficiency and generalization, attacks questions as diverse as whether to transfer knowledge, from whom, and in what form ( Taylor and Stone [n. d.] ; Bengio et al. 2009 ; Khetarpal et al. 2022 ) . Here, we are interested in a rather concrete flair of transfer learning: can an algorithm master a difficult task by moving through sub-tasks of increasing difficulty? This question can be particularly interesting for NE, which: a) can enable architecture search and, therefore, promote the emergence of structural features such as modularity, repetition, and hierarchy ( Gruau 1994 ; Gruau 1992 ; Risi et al. 2010 ; Stanley et al. 2009 ; Stanley and Miikkulainen 2002 ; Stanley and Miikkulainen 2003 ) , that have been linked to alleviating catastrophic forgetting ( Ellefsen et al. 2015 ) ; b) may maintain a population of solutions and employ explicit mechanisms for diversity-preservation ( Chalumeau et al. 2023 ; Such et al. 2018 ; Stanley and Miikkulainen 2002 ) , a capacity that has been linked to generalization but, we posit, could potentially also alleviate forgetting well-performing solutions. To investigate this, we introduce two benchmarks. The first one, stepping gates , contains tasks that require emulating the behavior of digital circuits consisting of different logic gates. Here, we instantiate two circuits: the classical N-parity , and our newly introduced Simple ALU , where the circuit supports switching within an instruction set of 16 simple, and composite logic functions, implementing a simplified version of an Arithmetic Logical Unit (ALU). N-parity aims at testing the ability to repeat identical modules, while the Simple ALU further requires repetition with modification (see Section 3.1 for a closer analysis of this benchmark). The second one, ecorobot , extends the physics-based engine Brax ( Freeman et al. 2021 ) with the presence of objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. In this way, we can disentangle the challenge of locomotion from a diversity of other behavioral challenges, such as maze navigation and obstructed locomotion, (as our results show this can shed light into limitations of different approaches). While both benchmarks can capture diverse challenges, here we focus on tasks that require transfer learning, which we impose by decomposing tasks into levels presented to the agent in order of increasing difficulty (we make a small exception to study avoidance of local optima in Section 5.2.3 .) Our empirical study aims to reveal how choices in the design of NE approaches influence their performance in the face of certain challenges. For this reason, we benchmark methods that differ across three orthogonal dimensions: a) whether they evolve the structure of the ANN or just the weights. For example, we show that NEAT ( Stanley and Miikkulainen 2002 ) , an algorithm that evolves both structure and weights, performs well across many transfer learning tasks (see sections 5.1 and 5.2 ); b) whether they are direct or indirect, i.e., whether evolution searches directly in the space of ANNs or a genotype to phenotype (GP) map is employed. Here, we show that HyperNEAT ( Stanley et al. 2009 ) , an indirect encoding, is disadvantaged at transfer learning but superior at avoiding local optima (see Section 5.2.3 ). c) whether they explicitly search for diversity or not. Here, we show that MAP-elites ( Mouret and Clune 2015 ; Chalumeau et al. 2023 ) , an algorithm excelling in deceptive tasks, is, perhaps surprisingly, at a disadvantage when we introduce a slight modification to the task that requires transfer learning (see Section 5.2.1 ). We also benchmark PPO, a state-of-the-art RL algorithm ( Schulman et al. 2017a ) , and goal-conditioned PPO ( Colas et al. 2022 ) , a simple variant designed to alleviate non-stationarity by conditioning the policy on the current task. Both methods exhibit sub-par transfer learning abilities. Finally, we show that many of the aforementioned benefits of NE disappear when the task remains unchanged, but the robotic morphology becomes more complex. Overall, our study confirms a variety of known results from the NE and RL literature, introduces new insights into differences between NE approaches, and offers new benchmarks aimed at supporting future work towards scaling up NE to richer behavioral challenges. We open-source our benchmarks in two online repositories 1 1 1 https://github.com/eleninisioti/ecorobot