PAL - Parallel active learning for machine-learned potentials
Chen Zhou, Marlen Neubert, Yuri Koide, Yumeng Zhang, Van-Quan Vuong, Tobias Schlöder, Stefanie Dehnen, Pascal Friederich · 2024
Read PAL - Parallel active learning for machine-learned potentials by Chen Zhou, Marlen Neubert, Yuri Koide, Yumeng Zhang, Van-Quan Vuong, Tobias Schlöder, Stefanie Dehnen, Pascal Friederich (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00401.txt.
Introduction
Machine-learned force fields can run fast atom simulations, but they are only as good as the data they train on. Chen Zhou and co-authors present PAL, a software library for growing that data in a smart loop, in a 2024 paper. The loop picks the most useful new cases, labels them with true physics, and retrains.
Older loops often need a human at each step and run one job at a time. PAL runs jobs in parallel on shared and spread-out machines, with no human in the loop. Users can swap in their own models, labelers, and pick rules. Tests on tasks from drug-like molecules to fluid flow showed strong speed gains. The tool suits big computing centers.
Catalog record
| CREATOR | Chen Zhou, Marlen Neubert, Yuri Koide, Yumeng Zhang, Van-Quan Vuong, Tobias Schlöder, Stefanie Dehnen, Pascal Friederich |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00401
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PAL - Parallel active learning for machine-learned potentials Chen Zhou, Marlen Neubert, Yuri Koide, Yumeng Zhang, Van-Quan Vuong, Tobias Schlöder, Stefanie Dehnen, Pascal Friederich Abstract Constructing datasets representative of the target domain is essential for training effective machine learning models. Active learning (AL) is a promising method that iteratively extends training data to enhance model performance while minimizing data acquisition costs. However, current AL workflows often require human intervention and lack parallelism, leading to inefficiencies and underutilization of modern computational resources. In this work, we introduce PAL, an automated, modular, and p arallel a ctive l earning library that integrates AL tasks and manages their execution and communication on shared- and distributed-memory systems using the Message Passing Interface (MPI). PAL provides users with the flexibility to design and customize all components of their active learning scenarios, including machine learning models with uncertainty estimation, oracles for ground truth labeling, and strategies for exploring the target space. We demonstrate that PAL significantly reduces computational overhead and improves scalability, achieving substantial speed-ups through asynchronous parallelization on CPU and GPU hardware. Applications of PAL to several real-world scenarios - including ground-state reactions in biomolecular systems, excited-state dynamics of molecules, simulations of inorganic clusters, and thermo-fluid dynamics - illustrate its effectiveness in accelerating the development of machine learning models. Our results show that PAL enables efficient utilization of high-performance computing resources in active learning workflows, fostering advancements in scientific research and engineering applications. 1 Introduction In many ML applications, the primary goal of data generation is to create a diverse dataset that comprehensively represents the relevant data distribution, ensuring stability and reliability during deployment 1 . In contrast, scientific applications often require ML models to operate in exploratory settings, where discovering novel and unseen inputs is expected or even desired. ML models trained on limited initial datasets often struggle to generalize effectively to newly encountered instances outside the training distribution. Continuously updating the model by labeling portions of the explored input space and retraining is inefficient, as it frequently adds redundant data points to the training set without significantly improving generalization accuracy. To address this challenge, active learning (AL) has been introduced as an efficient approach to building accurate and reliable models while minimizing queries to an oracle – a computational method or experimental process capable of providing ground truth labels 2 . By iteratively selecting the most informative instances for labeling, AL reduces the overall cost of training set generation and enhances model performance 3 , 4 , 5 . One particularly important application of active learning is developing new machine-learned potentials, which are often trained on quantum mechanical calculations to map three-dimensional atomic coordinates to corresponding total energies and forces 6 , 7 , 8 , 9 , 10 . These potentials are ML models typically based on neural networks 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 or Gaussian process models 10 , 20 , 21 , 22 . The advantage of ML potentials lies in their ability to perform molecular dynamics (MD) simulations with the accuracy of ab initio calculations but at only a fraction of the computational cost. As MD simulations often aim to explore the behavior and function of molecules and materials at the atomistic scale, constructing an initial dataset that fully represents all relevant geometries encountered during simulations is nearly impossible. Active learning approaches are therefore essential to update datasets and machine-learned potentials iteratively, ensuring both reliability and accuracy throughout the simulations 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 21 , 32 , 33 , 34 , 35 . Typically, active learning for machine-learned potentials is performed in a batch-wise manner, iterating through cycles of data labeling via quantum chemistry calculations, training ML models, and ML-accelerated simulations or other sampling strategies. Uncertainty quantification 36 , 37 , 22 is often used to select the most informative samples for subsequent iterations (see Figure 1 a). Despite the widespread use of active learning for machine-learned potential development, the current infrastructures and algorithms often fail to fully utilize modern computational resources, relying heavily on human intervention for tasks such as labeled data handling and model training initialization. Furthermore, the lack of parallelism introduces inefficiencies, as active learning is typically performed sequentially; for example, labeled data generation, model training, and exploration in molecular dynamics (MD) simulations are executed one after another. To address these challenges, we developed PAL, an automated, modular, and parallel active learning workflow (see Figure 1 b) with several key advantages: (1) the fully automatic workflow minimizes human intervention during execution; (2) the modular and highly adaptive design reduces the effort of (re-)implementing parts of the active learning workflow while allowing it to be extended to various tasks with diverse combinations of resources, data, and ML model types; (3) the decoupling of all AL modules enables data and task parallelism, facilitating simultaneous exploration/generation, labeling, and training tasks; and (4) PAL is implemented using the Message Passing Interface (MPI) in Python 38 , 39 , 40 , ensuring scalability and flexibility for deployment on both shared-memory systems (e.g., laptops, local workstations) and distributed-memory systems (e.g., high-performance computing clusters). We demonstrate the applicability and versatility of PAL across various applications of machine-learned potentials and other machine learning tasks for science and engineering beyond atomistic simulations. Our results highlight the scalability of PAL and the significant speed-ups achieved through asynchronous parallelization on CPU and GPU hardware. Figure 1: Comparison of (a) a conventional (serial) active learning and (b) our parallel active learning workflow PAL. a) Classical active learning workflow, in which different tasks , i.e. exploration of the input space using generation and prediction kernels, labeling of the samples using the oracle kernel, and training of the ML model, are performed iteratively and sequentially. b) PAL modularizes, decouples, and parallelizes data generation, oracle labeling, and ML training processes. 2 Description of the PAL workflow PAL is designed as a modular and scalable framework that orchestrates the components of an active learning workflow in a parallel computing environment. The architecture of PAL centers around five core modules, referred to as kernels: the prediction kernel, generator kernel, training kernel, oracle kernel, and controller kernel (see Figure 2 ). These kernels operate concurrently and communicate efficiently through the Message Passing Interface (MPI), enabling seamless integration and coordination on both shared- and distributed-memory systems. At a high level, the prediction kernel provides machine learning models that make predictions on input data generated by the generators. The generator kernel explores the target space by producing new data instances based on the predictions received. The training kernel retrains the machine learning models using newly labeled data to improve their performance iteratively. The oracle kernel acts as the source of ground truth labels, providing accurate labels for selected data instances that require further clarification. Overseeing these components, the controller kernel manages the workflow by coordinating communication, scheduling tasks, and handling data exchanges between kernels. This modular design allows users to customize and extend each kernel according to the specific requirements of their active learning scenarios. By decoupling the kernels and enabling them to operate in parallel, PAL maximizes computational