CHAMB-GA: A Containerized HPC Scalable Microservice-Based Framework for Genetic Algorithms
Felix Bonhoff · 2026
Read CHAMB-GA: A Containerized HPC Scalable Microservice-Based Framework for Genetic Algorithms by Felix Bonhoff (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27217.txt.
Introduction
Bonhoff presents CHAMB-GA, a framework that runs genetic algorithms with heavy simulations on everything from laptops to supercomputers. Genetic algorithms evolve candidate solutions through selection and mutation, but each fitness test may need a long simulation. Existing tool chains often lock users to one machine type, blocking smooth scaling. Containerised microservices solve that rigidity here. Deployment works on cloud clusters through orchestration and on HPC clusters through batch scheduling in this framework paper from 2026.
Users supply the genetic operators and the simulation backend separately. The two run decoupled on distinct hardware, coordinated by a central message broker. Fitness evaluations and evolutionary steps proceed in parallel across workers. Benchmarks show minimal overhead while scaling beyond thirty-five hundred CPU cores. A second test optimises high-voltage direct current line dispatch in the German grid. Migration between cloud and scheduler systems proves seamless, with mixed scaling and multi-stage workflows supported.
Catalog record
| CREATOR | Felix Bonhoff |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.27217
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CHAMB-GA: A Containerized HPC Scalable Microservice-Based Framework for Genetic Algorithms Felix Bonhoff Abstract a Institute of Climate and Energy Systems, Energy Systems Engineering (ICE-1), Forschungszentrum Jülich GmbH, Jülich 52425, Germany b RWTH Aachen University, Aachen 52062, Germany c RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen 52074, Germany d JARA-ENERGY, Jülich 52425, Germany e JARA-CSD, Jülich 52425, Germany Metaheuristic-based global optimization with embedded, long-running simulations is a computationally expensive process. To support various stages of development and execution, a seamless transition from personal computers to distributed clusters is desired, enabling execution across all computational scales. However, existing tool chains are often characterized by rigidity and hardware-bound constraints, which impede scalability and the integration of complex simulations. Bridging this gap, we present a c ontainerized H PC sc a lable m icroservice- b ased framework for g enetic a lgorithms with embedded simulations (CHAMB-GA). The deployment of the framework scales consistently across cloud infrastructure via container orchestration and HPC clusters via batch-scheduled parallel execution. Users provide the GA operators and simulation backend separately. The framework is designed to run these components in a distributed and decoupled manner, mapped to separate hardware. This approach ensures that the fitness evaluation and genetic operations are not managed within the same process and are utilizing distinct parts of the compute infrastructure. A central message broker coordinates asynchronous manager-worker communication between microservices, thereby parallelizing evolutionary operations and fitness evaluations. We demonstrate CHAMB-GA’s scalability, portability, and reproducibility, while facilitating the integration of external tools and complex simulations on benchmark and powerflow problems. The capabilities of CHAMB-GA are validated in a two-part approach: (i) a benchmark study demonstrating minimal overhead while scaling to over 3,500 CPU cores, and (ii) a dispatch optimization of High Voltage Direct Current (HVDC) lines in the German transmission grid, showing seamless migration from Kubernetes to SLURM, combined horizontal and vertical scaling, and integration of multi-stage workflows. Keywords : Container, HPC, Kubernetes, SLURM, Genetic Algorithm, Power Systems 1 Introduction Finding (near) optimal solutions to many real optimization problems is computationally challenging and may include embedded simulations, by choice or necessity. For instance, in the power systems domain, a prototypical simulation task is solving the AC powerflow equations. The computational effort for such simulations rapidly increases as network size expands ( 34 ; 3 ) , which becomes especially critical when many simulations have to be performed, for example in the context of security assessment, contingency analysis, or probabilistic studies. Furthermore, future approaches that integrate additional energy sectors, such as heating and gas ( 8 ; 20 ) , are expected to further increase model complexity, thereby making the associated simulation problems more computationally demanding. Solving optimization problems with embedded simulations is often approached with heuristic solvers ( 5 ; 16 ) , which require extensive sampling of the search space to limit the risk of being trapped in a suboptimal solution ( 22 ) . Many population-based heuristic algorithms, like genetic algorithms (GAs), inherently involve numerous independent fitness evaluations, making them specifically well-suited to run on parallel hardware. Inspired by natural selection, GAs are a class of meta heuristic optimization schemes that iteratively evolve a population of candidate solutions, called individuals, toward an optimal state. This process is driven by a few core mechanisms: evaluating the solution quality of each individual (fitness) via a problem-specific fitness function, selecting the most promising individuals (selection), and applying genetic operations (crossover and mutation) to generate the population of the subsequent iteration (generation) ( 36 ) . The growing accessibility and performance of parallel computing infrastructure, such as multicore systems, GPUs, and distributed architectures like HPC systems or cloud infrastructures, align with the increasing computational demands of large populations and complexity of fitness evaluations in GAs ( 9 ; 37 ) . In combination with being applicable to a wide variety of problems, GAs provide a flexible environment to utilize computational power and quickly adapt to different fitness evaluations and genetic operations. Particularly in research contexts, the ease of use, modularity, compatibility with complex simulation tools used for fitness evaluation, and integration with machine learning workflows to aid evolution ( 21 ) , make GAs a popular choice for solving a wide range of optimization problems ( 5 ; 16 ) . Many approaches and solutions arose to utilize this inherent parallelism, often closely coupled to current developments in computing hardware, such as multi-core processors, GPUs, cloud-computing and HPC systems. ( 12 ; 11 ) GAs are a widely used class of evolutionary algorithms (EAs), a generalization combining all stochastic search methods that mimic natural evolution. 4 defined a taxonomy of three types of parallelism in EAs: (i) The global model (manager-worker model) consisting of a single global population distributed across multiple worker processes. (ii) The coarse-grained model (island model) which partitions the population into multiple subpopulations, or islands, that evolve mostly independently of each other. Occasional migration of few individuals between islands to exchange knowledge enhances exploration. (iii) The fine-grained model where individuals undergo an evolutionary process on a structured grid, i.e., each individual only interacts with its neighbours or individuals from a neighbouring region, depending on the algorithm settings. To combine large-scale scalability with modular integration of different software stacks and simulation tools across container orchestration software and batch scheduling alike, we present a c ontainerized H PC sc a lable m icroservice b ased framework for g enetic a lgorithms with embedded simulations (CHAMB-GA). CHAMB-GA is a lightweight, open-source framework specifically designed to support scalable execution of evolutionary algorithms driven by computationally expensive fitness evaluations in various distributed compute environments. Utilizing containerized components enables independent development of separate tasks, such as propagation and fitness evaluation, and portability between different hardware, promoting reproducibility. Specifically, CHAMB-GA is a set of orchestration scripts that at its core employs a RabbitMQ ( 29 ; 30 ) message broker. Rather than adding more broker nodes, RabbitMQ scales vertically with the available CPU cores ( 29 ; 35 ) achieving good scalability while being deployable independently of the chosen orchestration software. The advanced routing capabilities of the broker ( 29 ) are used to internally handle parallel schemes such as the manager-worker scheme, the island model, or multi-stage workflows, as well as dynamically adjust worker counts and handling load balancing without redeployment or relying on a fixed pattern. All applications and tools run in containerized environments and can be combined to interact with each other in a scalable manner, enabling, e.g., multi-oracle analysis, see, e.g., 14 , or hyperparameter tuning. Various works have combined GA inherent parallelisation methods with distributed and parallel hardware ( 18 ; 2 ; 32 ) . We quantitatively compare CHAMB-GA to these approaches in Section 2. The remainder of this paper is structured as follows. Section 2 provides an overview of related works. Section 3 introduces the CHAMB-GA framework and describes key design choices and features. In Section 4, scalability and overhead are analyzed, followed by a power systems optimization problem demonstrating flexible resource assignment and a multi-stage workflow. Section 5 concludes with final remarks and possible future work. Table 1 : Comparison of evolutionary computing frameworks. Class Relevant frameworks Main features Existing frameworks distinctions to CHAMB-GA Monolithic • DEAP ( 6 ) • Geneva (