HyperNOs: Automated and Parallel Library for Neural Operators Research
Massimiliano Ghiotto · 2025
Read HyperNOs: Automated and Parallel Library for Neural Operators Research by Massimiliano Ghiotto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18087.txt.
Introduction
Massimiliano Ghiotto, writing alone in two thousand twenty-five, sets out HyperNOs, a Python library that makes the study of neural operators fast and hands-free. Neural operators are nets that learn maps between functions.
The library leans on a strong tune tool with parallel runs, to roam the space of settings for such nets in depth. It can train a model with a fixed count of terms, or on many sets and at many grades at once. Two known operator types are built in, and they hit top marks on known test sets. The face is made to be plain to use, and easy to stretch to new sets and new net forms.
Catalog record
| CREATOR | Massimiliano Ghiotto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 95 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18087
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HyperNOs: Automated and Parallel Library for Neural Operators Research Massimiliano Ghiotto Abstract This paper introduces HyperNOs, a PyTorch library designed to streamline and automate the process of exploring neural operators, with a special focus on hyperparameter optimization for comprehensive and exhaustive exploration. Indeed, HyperNOs takes advantage of state-of-the-art optimization algorithms and parallel computing implemented in the Ray-tune library to efficiently explore the hyperparameter space of neural operators. We also implement many useful functionalities for studying neural operators with a user-friendly interface, such as the possibility to train the model with a fixed number of parameters or to train the model with multiple datasets and different resolutions. We integrate Fourier neural operators and convolutional neural operators in our library, achieving state of the art results on many representative benchmarks, demonstrating the capabilities of HyperNOs to handle real datasets and modern architectures. The library is designed to be easy to use with the provided model and datasets, but also to be easily extended to use new datasets and custom neural operator architectures. 1 Introduction In recent years, the field of scientific machine learning has witnessed significant advances, particularly in the development of methods for solving partial differential equations (PDEs) and modelling complex dynamical systems. Traditional numerical methods, such as finite element analysis, isogeometric analysis [ 12 ] or spectral methods [ 4 ] , have long been the cornerstone of computational science and engineering. However, these methods often face challenges in scalability, computational cost, and generalization across different problem settings. In this context, Neural Operators (NOs) have emerged as a powerful paradigm that provides a data-driven approach to learning mappings between infinite-dimensional function spaces, allowing PDEs and other operator-related problems to be solved with unprecedented efficiency and flexibility. Neural operators, introduced as a generalization of neural networks to infinite-dimensional spaces, are designed to approximate operators that map input functions to output functions. Unlike traditional neural networks, which operate on finite-dimensional vectors, neural operators are capable of handling continuous functions, making them particularly suited for tasks such as solving PDEs, modelling physical systems, and performing surrogate modelling in high-dimensional spaces. The foundational work on neural operators, including architectures such as Deep Operator Networks (DeepONets, [ 25 ] ), Graph-based Neural Operators (GNOs, [ 21 ] ), and Fourier Neural Operators (FNOs, [ 22 ] ), has demonstrated their ability to learn solution operators for a wide range of PDEs. Many modifications of the above architectures have been progressively proposed, such as NOMAD [ 33 ] as a DeepONet modification, Multiple Graph Neural Operator [ 20 ] as a GNO modification, and many FNO modifications such as [ 34 , 37 , 24 , 23 ] . New architectures are also proposed, such as the Spectral Neural Operator (SNO, [ 7 ] ), the Convolutional Neural Operator (CNO, [ 31 ] ), and transformer-based operator learning [ 5 , 11 ] . The key advantage of neural operators lies in their ability to generalize across different discretizations of the domains, making them highly versatile for various applications, such as weather forecasting [ 28 ] or seismic waves [ 17 ] . By leveraging techniques from functional analysis, harmonic analysis, and deep learning, neural operators can be trained on data generated from traditional numerical PDE approximation or with physical measures and subsequently used to predict solutions for new inputs. Moreover, it is theoretically demonstrated (see [ 15 , 26 ] for DeepONets, [ 13 ] for FNOs, [ 31 ] for CNOs and [ 16 ] for NOs in general), that neural operators can approximate the solution operators for many families of PDEs, enabling rapid and accurate predictions for new instances of the problem. This capability is particularly valuable in scenarios where traditional numerical methods are computationally expensive or infeasible, such as for high-dimensional problems, many query problems (e.g., uncertainty quantification or inverse problems) and real-time simulations. The versatility of neural operators has led to their adoption in a wide range of scientific applications and engineering domains. Despite their impressive capabilities, training neural operators presents several challenges. One of the primary challenges is the selection of appropriate hyperparameters, which play a critical role in determining the performance and generalization ability of the model. Hyperparameters include architectural choices (e.g., the number of layers, the type of activation functions, and the size of the hidden layers), optimization parameters (e.g., learning rate, batch size, and optimizer type), and regularization parameters (e.g., learning rate scheduler and weight decay). Because the hyperparameter space like grid search is often large and complex, manual hyperparameter tuning is a labour-intensive and error-prone process that often requires domain expertise, and banal search spaces are too computationally expensive. Automated hyperparameter tuning is a fundamental component of modern machine learning workflows, especially for complex models such as neural operators. The process involves systematically searching the hyperparameter space to identify configurations that maximize model performance, as measured by a predefined loss function. Automated hyperparameters tuning provides several key benefits: • Improved Model Performance: The performance of machine learning models, especially neural operators, is highly sensitive to the choice of hyperparameters. Automated tuning identifies configurations that maximize model performance on validation data, leading to better generalization and robustness. This is particularly important for neural operators, where the relationship between hyperparameters and performance is often non-intuitive and problem-dependent. • Efficiency and Automation: Automated tuning algorithms systematically explore the hyperparameter space, reducing the need for manual intervention. Manual hyperparameter tuning is time-consuming and labour-intensive, requiring extensive trial and error and some domain knowledge, this automation saves time and allows researchers to focus on higher-level tasks, such as model design and interpretation of results. • Adaptability to Different Domains: Neural operators are used in a wide variety of domains and applications, each with its own unique challenges and requirements. Automated tuning can adapt to these diverse contexts by optimizing hyperparameters for specific tasks, datasets, and performance metrics. • Scalable and Parallel: As models and datasets grow in size and complexity, the hyperparameter optimization cost grows exponentially. Manual tuning becomes impractical for large problems due to the sheer number of possible configurations. Automated tuning methods efficiently handle large and high-dimensional search spaces using parallel computing and concentrating on promising hyperparameters configuration and automatically stops bad trials. • Reproducibility: Reproducibility is a cornerstone of scientific research, ensuring that results can be independently verified and validated. We provide a systematic approach that stores hyperparameter configurations and trained models to ensure reproducible results. • Human and Machine Efficiency: As mentioned above, human intuition is limited in navigating complex hyperparameter spaces, especially for high-dimensional models such as neural operators. We show numerically that by offloading the tuning process to machines, trained models are not affected by local minima due to suboptimal hyperparameter choices. To address this challenge, we introduce HyperNOs , a PyTorch library designed to streamline and automate the process of exploring neural operators, with a special focus on hyperparameter optimization for comprehensive and exhaustive exploration. HyperNOs takes advantage of state-of-the-art optimization algorithms and parallel computing implemented in the Ray-tune library to efficiently explore the hyperparameter space of neural operators. We also implement many useful functionalities for studying neural operators with a user-friendly interface. Furthermore, we provide a set of code examples with different datasets