Ecological Neural Architecture Search
Benjamin David Winter 1 , William J. Teahan 2 · 2025
Read Ecological Neural Architecture Search by Benjamin David Winter 1 , William J. Teahan 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10908.txt.
Introduction
Hunting the best shape for a neural net by breed and test is strong, but the hunt has its own dials to set by hand. Benjamin David Winter and William Teahan fold those dials into the nets in a 2025 study. Each net in the pool carries its own birth, change, and clone rates in its form.
Nets that grab more stock when they are fit can then steer the hunt as it runs. On four two class test sets, the self tuned hunt beat kin hunts on speed and most oft on score. Run time fell by near a fifth. The team calls the breed plan a self ruled way to grow nets with no hand set dials.
Catalog record
| CREATOR | Benjamin David Winter 1 , William J. Teahan 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-10908
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ecological Neural Architecture Search Benjamin David Winter 1 , William J. Teahan 2 Abstract When employing an evolutionary algorithm to optimize a neural network’s architecture, developers face the added challenge of tuning the evolutionary algorithm’s own hyperparameters—population size, mutation rate, cloning rate, and number of generations. This paper introduces Neuvo Ecological Neural Architecture Search (ENAS), a novel method that incorporates these evolutionary parameters directly into the candidate solutions’ phenotypes, allowing them to evolve dynamically alongside architecture specifications. Experimental results across four binary classification datasets demonstrate that ENAS not only eliminates manual tuning of evolutionary parameters but also outperforms competitor NAS methodologies in convergence speed (reducing computational time by 18.3%) and accuracy (improving classification performance in 3 out of 4 datasets). By enabling ”greedy individuals” to optimize resource allocation based on fitness, ENAS provides an efficient, self-regulating approach to neural architecture search. I Introduction Neural Architecture Search (NAS) was developed to search a search space for optimal network architectures on a task-specific basis. EA-inspired NAS models such as Neuvo NAS+ and Neuvo GEAF do this. However, they come with a downside of adding more hyperparameters to the search space, the evolutionary parameters such as the population size, cloning rate, mutation rate and the number of generations to be specific. It can be assumed that an optimum combination of evolutionary parameters exists, and that likely this should be performed on a task-specific basis as well. One could imagine a complex dataset where only few minima exist and radical changes to the network’s architecture are needed. This type of task would potentially benefit from a higher mutation rate. However, this is impossible to change during runtime as evolutionary parameters in traditional EAs are static and set before program compilation. A further problem of an EA model developed with static evolutionary hyperparameters is that the process will run for the maximum number of generations regardless whether an optimal network has been found resulting in computational resources being wasted. Finally, imagine if the problem requires more diversity amongst the population. One could simply increase the number of individuals in the population. However, what if after n generations, a fit individual has been found? This would result in a reduction in diversity being desired, paired with a higher cloning rate and a higher mutation rate to fine tune the fit individual’s genes. Furthermore, a larger population requires more computing resources but to retrain these individuals when it is not needed would be wasteful of vital resources. Under the current implementations of evolutionary algorithms, this dynamic evolutionary parameter change has not previously been investigated. ENAS addresses these issues by including the evolutionary parameters in the individual’s genomes, allowing dynamic evolutionary parameters throughout the evolution, including global and local hyperparameters such as: population size, mutation rate, cloning rate and the number of generations. Our approach is inspired by ecological phenomena in nature, particularly species that optimize resource allocation even at the expense of population diversity. A striking example is the Christmas Island red crab (Fig. 1 ), which devours its own offspring to preserve resources for itself—similarly, our ”greedy individuals” can reduce population size to allocate more computational resources to the fittest architectures. Figure 1 : A Christmas Island red crab devouring its own offspring, illustrating natural resource optimization at the expense of population diversity. This natural resource optimization strategy parallels computational challenges in distributed systems and resource-constrained environments. Just as the red crab sacrifices population diversity to ensure individual survival, modern computational systems often implement resource allocation algorithms that dynamically prioritize critical processes by reducing resources for lower-priority tasks. Cloud computing platforms use similar principles when scaling resources up or down based on demand, effectively ’cannibalizing’ idle resources to redirect them where they’re most needed. ENAS applies this ecological principle to neural architecture search by allowing fitter individuals to influence global evolutionary parameters, potentially reducing population diversity but optimizing computational resource allocation toward the most promising architectural candidates. This paper describes a novel methodology – Ecological Neural Architecture Search (ENAS). ENAS addresses these issues by including the evolutionary parameters in the individual’s genomes, allowing dynamic evolutionary parameters throughout the evolution, including global and local hyperparameters such as: population size, mutation rate, cloning rate and the number of generations. II Background and Related Work Neural Architecture Search (NAS) has emerged as a crucial research area that aims to automate the design of optimal neural network architectures for specific tasks. Early NAS approaches such as those by Zoph and Le [ 1 ] utilized reinforcement learning to search for architectures, but were computationally intensive. Evolutionary algorithms (EAs) have since become popular for NAS due to their ability to efficiently explore large search spaces. Several EA-based NAS approaches have been proposed in recent years. Real et al. [ 2 ] introduced an evolutionary algorithm for image classification tasks that utilized tournament selection and age-based regularization. Similarly, Liu et al. [ 3 ] proposed a hierarchical representation that enabled the search for architectures with skip connections and multiple branches. The challenges of hyperparameter optimization for both network architecture and evolutionary algorithms themselves have been highlighted by several researchers. Feurer and Hutter [ 4 ] discussed the nested hyperparameter optimization problem, where an algorithm’s hyperparameters must be optimized for each task. Bergstra and Bengio [ 5 ] demonstrated that random search could be more efficient than grid search for hyperparameter optimization, suggesting that the search space for optimal hyperparameters is complex. Agent-based evolutionary hyperparameter optimization approaches have been explored by Esmaeili et al. [ 6 ] , but these typically localize evolutionary parameters to individual agents, rather than including them in the global optimization process. Unlike our approach, previous work has generally treated evolutionary parameters as static values set before algorithm execution. The concept of dynamic parameter adjustment in EAs has been explored in limited contexts. Meyer-Nieberg and Beyer [ 7 ] reviewed self-adaptive parameter control in evolutionary algorithms, but primarily focused on mutation step sizes rather than comprehensive evolutionary parameters. Eiben et al. [ 8 ] provided a taxonomy of parameter control in evolutionary algorithms, distinguishing between parameter tuning (setting parameters before the run) and parameter control (changing parameters during the run). Our approach, Ecological Neural Architecture Search (ENAS), differs from previous work by incorporating the evolutionary parameters directly into the individuals’ genomes, allowing for dynamic adjustment of global evolutionary parameters during the evolutionary process. This approach is inspired by ecological principles where populations adapt their reproductive strategies based on environmental conditions. III Methodology The ENAS algorithm currently piggybacks on Neuvo’s NAS+ approach to be run. This approach is a Genetic Algorithm based approach where each individual gene in a candidate solution’s genotype represents a different network architecture feature. Neuvo’s ENAS methodology involves appending four more ‘ecological’ genes to a candidate solution’s phenotype, as seen in Listing 1 . ⬇ 1 { ‘ hidden_layers ’ : 2 , 2 ‘ nodes ’ : 4 0 , 3 ‘ activation functions ’ : [ ’ relu ’ , ’ relu ’ , ’ relu ’ , ’ sigmoid ’ ] , 4 ‘ optimiser ’ : ’ Adam ’ , 5 ‘ number of epochs ’ : 5 0 , 6 ‘ batch size ’ : 2 , 7 ‘ mutation rate