Neural architecture search using network embedding and generative adversarial networks Authors: Morteza Yousefi, Vahid Mehrdad, Mohammad Bagher Dowlatshahi Published: 2025 ABSTRACT Surrogate models are used by recently proposed algorithms as a means of forecasting neural architecture performance. Rather than training the network from scratch, which speeds up evaluation of performance in the search for neural architecture. However, collecting a sufficient number of labeled architectures for training surrogate models is a time-consuming process. We suggest a surrogate-assisted swarm optimization algorithm with network embedding for neural architecture search, as well as a generative adversarial networks for augmentation data (GNE-NAS) to improve the performance of surrogate models with limited training data. In this case, each architecture is meaningfully represented using an unsupervised learning technique. In the embedding space, architectures with a greater degree of structural similarity are positioned closer together. This proximity facilitates the training of surrogate models. Prior to training the surrogate model, we employ a generative adversarial network for data augmentation. This approach enhances the robustness of the surrogate model and concurrently reduces the need for a large number of real evaluations. The surrogate model achieves comparable or better performance when network embedding is applied, as demonstrated by experimental results on two distinct NASBench search spaces. Our proposed method, GNE-NAS, has been shown to outperform other state-of-the-art neural architecture search algorithms. Introduction Deep neural networks (DNNs) have achieved remarkable success in a wide range of artificial intelligence tasks, including image classification, object detection, and natural language processing1–3. Their performance depends not only on the optimization of network weights but also critically on the underlying architecture design. Traditionally, creating high-performing architectures has required substantial domain expertise and iterative experimentation. To reduce this dependence on manual design, Neural Architecture Search (NAS) has emerged as a powerful paradigm, formulating architecture design as an optimization problem over a predefined search space and aiming to discover network structures that maximize validation performance after training. Existing NAS approaches can be broadly grouped into three categories: reinforcement learning RL-based, evolutionary algorithm EA-based, and gradient-based methods. RL-based NAS employs a controller that samples candidate architectures and learns to improve its proposals based on performance rewards. EA-based NAS treats architectures as individuals in a population, generating new ones via evolutionary operations like selection, crossover, and mutation. In contrast, gradient-based NAS relaxes the discrete nature of the search space into a continuous form, allowing architecture parameters and network weights to be optimized jointly via gradient descent. While EAs are known for their strong exploration capabilities4, their reliance on full training of each candidate makes them computationally costly, for instance, LargeEvo5 requires more than 3000 GPU-days for a single search. Surrogate-Assisted Evolutionary Algorithms (SAEAs) address this computational bottleneck by replacing expensive full training with performance prediction through surrogate models. Examples include end-to-end performance predictors6, adaptive switching between multiple surrogates7, and LSTM-based estimators8. Although such strategies reduce computational expense, supervised surrogate models still require large sets of well-trained architectures as labeled data, limiting their scalability. To address this, recent methods have incorporated unsupervised or self-supervised learning to obtain architecture embeddings that reduce labeled data demands, such as Arch2vec9, NASGEM10, and SS-RL/SS-CCL11. However, these works often impose restrictive assumptions on the embedding distribution or neglect operation-level details that significantly influence performance. Alternative embedding techniques, such as graph convolutional networks, contrastive graph objectives, and transformer-based graph encoders, have also been explored to produce richer representations of architectural graphs12–14. Another promising direction is online learning with infill criteria, in which the surrogate model is updated incrementally as new evaluations are obtained. Infill criteria select candidate architectures that balance exploration of new regions with exploitation of promising areas, guided by predicted performance and uncertainty15,16. In parallel, environmental selection strategies help maintain diversity in the search population while preserving high-quality solutions, supporting both efficient convergence and robustness against premature stagnation. In this paper, we propose a surrogate-assisted swarm intelligence algorithm for NAS that integrates three key innovations:Graph2vec-based architecture embedding: We apply the graph2vec algorithm to produce dense, informative embeddings of candidate architectures, mapping structurally similar networks close together in the latent space. This enables more efficient and accurate search operations compared to navigating the discrete architecture space directly.GAN-based data augmentation for surrogate models: We employ generative adversarial networks to synthesize high-quality artificial architectures, enriching the surrogate’s training set. This improves prediction robustness and accuracy, especially under limited real-evaluation data.Online learning-based infill criterion: We design a dual-objective infill criterion that considers both predicted accuracy and model uncertainty, guiding the search toward promising architecture regions while preserving exploration. This criterion is integrated into the swarm search to dynamically update and improve the surrogate. Graph2vec-based architecture embedding: We apply the graph2vec algorithm to produce dense, informative embeddings of candidate architectures, mapping structurally similar networks close together in the latent space. This enables more efficient and accurate search operations compared to navigating the discrete architecture space directly. Graph2vec-based architecture embedding: We apply the graph2vec algorithm to produce dense, informative embeddings of candidate architectures, mapping structurally similar networks close together in the latent space. This enables more efficient and accurate search operations compared to navigating the discrete architecture space directly. GAN-based data augmentation for surrogate models: We employ generative adversarial networks to synthesize high-quality artificial architectures, enriching the surrogate’s training set. This improves prediction robustness and accuracy, especially under limited real-evaluation data. GAN-based data augmentation for surrogate models: We employ generative adversarial networks to synthesize high-quality artificial architectures, enriching the surrogate’s training set. This improves prediction robustness and accuracy, especially under limited real-evaluation data. Online learning-based infill criterion: We design a dual-objective infill criterion that considers both predicted accuracy and model uncertainty, guiding the search toward promising architecture regions while preserving exploration. This criterion is integrated into the swarm search to dynamically update and improve the surrogate. Online learning-based infill criterion: We design a dual-objective infill criterion that considers both predicted accuracy and model uncertainty, guiding the search toward promising architecture regions while preserving exploration. This criterion is integrated into the swarm search to dynamically update and improve the surrogate. Related works In recent years, EAs have gained popularity in NAS. Among these, the Genetic Algorithm (GA) is the most popular optimizer in EA-based NAS. Additionally, Particle Swarm Optimization (PSO) has emerged as another prominent algorithm in this field. LargeEvo5 is an evolutionary algorithm used in NAS. It employs binary tournament selection for mating, each pair of architectures compete, and the better-performing one becomes a parent. Genetic operations like crossover and mutation create new architectures in each generation. The algorithm maintains diversity in the population and removes older individuals to introduce fresh genetic material. Genetic CNN17 utilizes genetic operations to evolve and create new neural network architectures in each generation. The fixed-length binary representation allows for efficient manipulation of the architecture’s components. This approach aims to automatically discover architectures that excel in specific tasks, eliminating the need for manual design. REA4 maintains population diversity by eliminating the oldest individuals, which contributes to achieving state-of-the-art performance. Cartesian genetic programming for convolutional neural networks, CGP-CNN18,19, introduces an innovative approach to architectural exploration in Convolutional Neural Networks (CNNs). Utilizing the capabilities of Cartesian genetic programming, this method employs an evolutionary search algorithm to generate and optimize diverse CNN architectures iteratively. The distinctive strengths of CGP-CNN lie in its ability to navigate a vast architectural space with flexibility and efficiency while minimizing computational overhead. AutoML-Zero20, employs a distinctive approach in NAS by utilizing fundamental mathematical operations as the foundational building blocks for designing neural network architectures. In contrast to traditional methods that rely on expert-crafted layers, AutoML-Zero starts with elementary operations and leverages an evolutionary process to generate and refine architectures iteratively. This unique strategy demonstrates the algorithm’s ability to autonomously discover architectures from scratch, significantly reducing reliance on pre-defined, human-designed components. DE-NAS21 utilizes Differential Evolution (DE) in NAS, adapting it to handle the discrete and categorical nature of architectures. Building on this line of research, Han et al.22 introduced SaDENAS, a self-adaptive differential evolution algorithm that tunes its control parameters based on the search stage. This strategy accelerates convergence while maintaining strong exploration capability, and their experiments on multiple NAS benchmarks reported clear gains over conventional DE-NAS variants. In a related hybrid direction, Xue et al.23 proposed GENAS, a gradient-guided evolutionary NAS framework. By incorporating gradient information into evolutionary operators, GENAS effectively navigates toward promising regions of the search space while preserving population diversity. Their results show consistent improvements compared with both pure evolutionary and pure gradient-based approaches. Extending recent advances to gradient-based NAS, Xue et al.24 developed a differentiable NAS method with a self-adaptive weighting scheme based on dual-attention mechanisms. This approach dynamically balances the influence of candidate operations during architecture search, yielding more stable convergence and better performance on CIFAR-10 and ImageNet than standard DARTS. Within the domain of NAS, PSO can be used to optimize neural network architectures by representing the network design as a particle in the search space. The algorithm iteratively updates the position of each particle (network architecture) by evaluating its performance relative to the best-performing particle of the swarm, this process aims to identify the architecture that achieves the best validation performance25. The application of PSO in NAS is valuable due to its ability to handle the vast search space and complex objective functions involved in designing neural networks. It balances exploration and exploitation, crucial for avoiding local optima and ensuring convergence to a global optimum26. Furthermore, the flexibility of PSO allows it to be adapted for various NAS tasks, such as feature selection, hyperparameter tuning, and layer configuration, making it a versatile tool for improving neural network performance across different datasets and problem domains27. In summary, PSO is a promising approach for NAS, offering a systematic method to explore and optimize neural network architectures, which can result in the development of more efficient and accurate models for a wide range of applications25. The application of Ant Colony Optimization (ACO) to NAS is promising because it can potentially discover high-performing architectures that might not be easily found through manual design or other search strategies. The algorithm can gradually search for the best architectures by covering the entire search space in tiny, incremental steps28. The Firefly Optimization Algorithm (FA) has been applied in various domains, including optimizing Multilayer Perceptron (MLP) structures for classification tasks. For instance, a Variable Step Size Firefly Optimization Algorithm (VSFA) has been employed to optimize the weights and biases of neurons in an MLP model, with the goal of minimizing the mean square error of the network output29. Encoding schemes The impact of encoding schemes on search performance is a complex topic that has been explored in various research contexts. Encoding schemes play a vital role in representing data in a storage-efficient format and conducive to fast retrieval and processing. In the context of search performance, encoding schemes can significantly affect the speed and accuracy of search algorithms. Adjacency matrix encoding is a straightforward representation of graph data, where an edge in a graph is represented as either present or absent by a two-dimensional matrix. This encoding is particularly useful for dense graphs and allows quick lookups of edge existence. However, the adjacency matrix can be space-inefficient for sparse graphs, as it requires storage for all possible node pairs, regardless of whether an edge exists30. Path-based encodings, on the other hand, represent paths or sequences in a graph, which can be more space-efficient for sparse graphs. These encodings can be optimized for specific types of searches, such as shortest-path queries in transportation networks31. In NAS, path-based encoding encodes every path that connects an input to an output by treating the neural network as a Directed Acyclic Graph (DAG). This method contrasts with adjacency matrix encoding and has been shown to reduce feature dependency and improve the performance of neural predictors32. BANANAS33 is one of the algorithms that adopts path-based encoding, and it is noted for its innovative approach in this area. The research on path-based encoding and its advantages over traditional adjacency matrix encoding is supported by various studies. For instance, the Directed Acyclic Graph Variational Autoencoders (D-VAE) encode an input graph using a graph neural network, which computes encodings by following the structure of the graph, a method similar to the recursive neural networks employed by RTG-AE34. Moreover, Graph Neural Networks (GNNs) are highly relevant to the discussion of path-based encoding. To process complex graph-structured data, GNNs, a class of deep learning algorithms, iteratively update the representation of each node by aggregating information from its neighboring nodes35. This iterative process is similar to path-based encoding, as it approaches emphasizes the exploration of paths within the graph. The effectiveness of path-based encoding in NAS is further highlighted by the use of surrogate models in NAS. These models, such as GATES, process neural architecture information and map it into a continuous embedding space, potentially representing the networks data36. The concept of Position-Aware Path-based Encoding (PAPE) in NAS represents a novel approach to encoding neural network architectures. Unlike traditional path-based encoding, which may only consider the type of operations in a path, PAPE also records the position of each operation. This allows for a more detailed representation of the architecture’s structure, potentially leading to more effective search and optimization in NAS. Within the domain of NAS, encoding schemes like PAPE are crucial because they influence how architectures are represented and manipulated during the search process. The encoding needs to capture the essence of the architecture topology and operations to allow for efficient search and performance prediction37. Inspired by biological processes, Action Command Encoding (ACEncoding) has been proposed38, which encodes these commands into a variable-length sequence using seven action commands made up of three integers. This technique enables the representation of more intricate neural architectures, suggesting a parallel with the gene expression process in biological systems. The ACEncoding scheme is particularly notable for its ability to encode a sequence of architectural decisions, similar to how gene expression encodes the development of biological organisms. The flexibility and depth of representation offered by this method could potentially lead to the discovery of more nuanced and effective neural network architectures, especially when considering the vast and complex search spaces involved in NAS39. Performance estimation strategy NAS is critical in automated machine learning, particularly for designing efficient neural networks. Evaluating new architectures is a significant bottleneck in NAS due to the high computational cost of training and validating each candidate model. To address this, numerous techniques have been developed to accelerate the evaluation process. One such method is low-fidelity estimation, which involves training models on a subset of the data or for fewer epochs to quickly estimate their performance40,41. Although this method can drastically reduce the computational resources requirements, the resulting performance estimations might be less precise5,42. Another method is the weight-sharing mechanism, which entails training a supernet that includes every possible architecture in the search space. Individual architectures can be evaluated without training from scratch by inheriting weights from the supernet. This method can lead to faster evaluations but may introduce biases due to the shared weights43. Surrogate model techniques involve training a predictive model, such as a Gaussian process or a neural network, to predict the performance of architectures without actually training them. This approach can be very efficient, especially when combined with transfer learning or meta-learning techniques, but the accuracy of the predictions depends on the surrogate model’s quality44. One important tactic to lower computational costs and expedite the evaluation of novel neural network architectures is the weight-sharing mechanism in one-shot architecture search. The literature provides various approaches to implementing weight sharing, each with advantages and potential drawbacks. SMASH45 is an early approach that utilizes a hypernetwork to dynamically generate weights for different architectures, which are then evaluated based on their performance46. This method allows for a rapid assessment of numerous architectures without needing to train each individually. One-shot NAS methods, such as the one proposed by47, employ dynamic dropout rates during supernet training to mitigate the weight co-adaptation problem. This involves randomly zeroing out a subset of operations, which encourages the network to become more robust to architectural changes. SPOS48 assumes that each architecture is a single path within the supernet and uses uniform path sampling during training. This method aims to alleviate the weight co-adaptation issue by ensuring that each path is treated equally during the training process46. FairNAS49 addresses the inherent unfairness in supernet training by ensuring that each block within the network is activated and updated only once, promoting fairness in evaluating candidate architectures46. Landmark Regularization50 introduces a regularization term that leverages stand-alone performance to guide the supernet training, providing a way to incorporate external knowledge into the training process51. DARTS52 adopts a gradient-based optimization method that simultaneously optimizes the weights and the architecture, allowing for a more integrated and efficient search process53. Preliminaries: graph2vec Graph representation learning has emerged as a crucial area of research in machine learning, particularly for tasks involving graph-structured data such as social networks, molecular structures. Traditional approaches, such as graph kernels, rely on handcrafted features to measure graph similarity, which often suffer from poor generalization and high computational costs. To address these limitations, graph2vec54 introduces a novel unsupervised framework for learning distributed representations of entire graphs, enabling efficient and scalable graph analytics. The core idea behind graph2vec is inspired by document embedding models in NLP, particularly the doc2vec model55. In this analogy, an entire graph is treated as a document, and its rooted subgraphs are considered as words that compose the document. By leveraging this analogy, graph2vec extends the skip-gram model to learn embeddings for graphs in an unsupervised manner. Specifically, the algorithm samples rooted subgraphs around each node in a graph, assigns unique labels to these subgraphs using the Weisfeiler-Lehman (WL) relabeling strategy, and then applies a skip-gram model to learn graph embeddings. This process ensures that structurally similar graphs are mapped to nearby points in the embedding space, preserving their inherent properties. One of the key advantages of graph2vec is its ability to capture structural equivalence in graphs. Unlike traditional methods that rely on linear substructures such as random walks or shortest paths, graph2vec considers non-linear substructures (i.e., rooted subgraphs), which provide a richer representation of the graph topology. This approach allows graph2vec to outperform existing substructure-based embedding methods and achieve competitive results compared to state-of-the-art graph kernels. Moreover, the embeddings produced by graph2vec are task-agnostic, meaning that they can be applied to various downstream tasks such as graph classification, clustering, and even as initializations for supervised learning models. The training process of graph2vec involves two main steps: (1) extracting rooted subgraphs using the WL relabeling strategy, and (2) learning graph embeddings through a skip-gram model with negative sampling. Negative sampling is employed to reduce computational complexity by updating only a small subset of negative samples during each training iteration. This makes graph2vec computationally efficient while maintaining high accuracy. The resulting embeddings are optimized using Stochastic Gradient Descent (SGD), ensuring that the learned representations are both meaningful and scalable. Wasserstein GAN with gradient penalty (WGAN-GP) Generative adversarial networks are a class of machine learning systems composed of two neural networks-a discriminator and a generator that are trained simultaneously to produce realistic data. The generator wants to create data indistinguishable from real data, while the discriminator wants to tell real data from generated data56. Goodfellow et al. (2014) recommended implementing a non-saturating loss for the generator, which provides much stronger gradients56. Despite their success, traditional GANs often suffer from training instability and mode collapse. Wasserstein GANs (WGANs) address the instability issues of traditional GANs by using the Wasserstein distance, which provides better theoretical properties and more stable training dynamics. The Wasserstein distance measures the cost of transforming one distribution into another, and in WGANs, the critic (replacing the discriminator) is trained to estimate this distance57. The objective function for WGANs is given by:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathop {\min }\limits_{G} \mathop {\max }\limits_{{D \in {\mathcal{D}}}} {\mathbb{E}}_{{x\sim {\mathbb{P}}_{r} }} \left[ {D\left( x \right)} \right] - {\mathbb{E}}_{{\tilde{x}\sim {\mathbb{P}}_{g} }} \left[ {D\left( {\tilde{x}} \right)} \right],$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal{D}$$\end{document} is the set of 1-Lipschitz functions, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb{p}}_{r}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb{p}}_{g}$$\end{document} represent the real and generated data distributions, respectively. However, enforcing the Lipschitz constraint through weight clipping, as done in standard WGANs, can lead to optimization difficulties and poor sample quality. To overcome these limitations, WGAN-GP58 introduces a gradient penalty term as an alternative to weight clipping. The gradient penalty ensures that the norm of the critic gradient with respect to its input is close to 1, effectively enforcing the Lipschitz constraint. The modified loss function is:2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} L_{D} = & {\mathbb{E}}_{{\tilde{x}\sim {\mathbb{P}}_{g} }} \left[ {D\left( {\tilde{x}} \right)} \right] - {\mathbb{E}}_{{x\sim {\mathbb{P}}_{r} }} \left[ {D\left( x \right)} \right] \\ & \quad + \lambda {\mathbb{E}}_{{\hat{x}\sim {\mathbb{P}}_{{\hat{x}}} }} \left[ {\left(\Vert {\nabla_{{\hat{x}}} \left( {D\left( {\hat{x}} \right)} \right)\Vert _{2} - 1} \right)^{2} } \right], \\ \end{aligned}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda$$\end{document} is the penalty coefficient (typically set to 10), and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb{P}}_{{\hat{x}}}$$\end{document}.is the distribution of samples drawn uniformly along straight lines between pairs of points from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb{p}}_{r}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb{p}}_{g}$$\end{document}. This sampling strategy is motivated by the fact that the optimal critic has a gradient norm of 1 along these lines. In practice, WGAN-GP avoids using batch normalization in the critic because it introduces dependencies between samples in a batch, which conflicts with the gradient penalty term. Instead, layer normalization is recommended as a drop-in replacement. The critic should be a neural network with sufficient capacity to approximate the Wasserstein distance effectively. Training is typically performed using optimizers like Adam with a learning rate of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${10}^{-4}$$\end{document}, and the gradient penalty is applied to random interpolated samples during each training iteration. The overall training and generation procedure of the proposed WGAN-GP framework is summarized in Algorithm 1. In the context of NAS, we adopt WGAN-GP as a generative backbone to augment the architecture search space with high-quality embeddings. Unlike standard GANs, WGAN-GP leverages the Wasserstein-1 distance to achieve highly stable training and significantly reduce mode collapse, thereby producing diverse architecture candidates. Generator network (G) The generator is implemented as a multi-layer perceptron (MLP) that maps a 128-dimensional noise vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$z\sim N(0, I)$$\end{document} into a continuous embedding representing a candidate architecture. The network consists of three fully connected layers (dimensions: 128 → 256 → 128), each followed by ReLU activation, and a linear output layer matching the dimensionality of the architecture embedding. Optional batch normalization can be applied to further stabilize training. Critic network (D) The critic is also realized as an MLP, receiving either real or generated architecture embeddings and outputting a scalar score proportional to their Wasserstein estimate. The architecture consists of fully connected layers 128 → 256 → 128 with LeakyReLU activations (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha =0.2$$\end{document}) and a single linear output neuron without activation. Training objective In addition to the critic loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${L}_{D}$$\end{document} (Eq. 3), the generator is optimized with:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_{G} = - {\mathbb{E}}_{{z\sim {\mathcal{N}}\left( {0, I} \right)}} \left[ {D\left( {G\left( z \right)} \right)} \right],$$\end{document}which encourages the generator to produce samples that maximize the critic’s score. Following the original WGAN-GP training protocol, the critic is updated for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${n}_{critic}=5$$\end{document} steps per generator update. Both networks are trained with the Adam optimizer (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$lr={10}^{-4}, {\beta }_{1}=0.5, {\beta }_{2}=0.9$$\end{document}), a batch size of 64, and up to 300 epochs. After generation, each synthetic embedding is validated against the target NAS search space, retaining only valid architectures for downstream training. By integrating WGAN-GP in the NAS pipeline, we achieve a generative model capable of producing diverse, high-quality architecture candidates while maintaining training stability, an essential factor in expanding and effectively exploring the search space56–58. Motivation of using PSO Particle Swarm Optimization (PSO) is a population-based stochastic optimization technique inspired by the social behavior of birds flocking. Since its introduction by Kennedy and Eberhart in 1995, PSO has gained significant popularity due to its simplicity, efficiency, and ability to solve complex optimization problems across various domains. The primary motivation for using PSO in this study stems from its robust performance in handling high-dimensional, non-linear, and multi-modal optimization problems, which are often encountered in real-world applications. The PSO algorithm offers several advantages compared to other optimization algorithms, contributing to its popularity as a solution for various uses. The main benefits of PSO are as follows: Easy implementation: PSO is relatively easy to understand and implement compared to other optimization algorithms. Its straightforward concept and low coding effort requirements make it accessible to many users. Few algorithm parameters: PSO has a small number of algorithm parameters that need to be tuned, such as the population size and the inertia weight. This simplicity facilitates the application of the algorithm to various problems and reduces the complexity of parameter tuning. Effective parallel operations: The PSO algorithm is well-suited for parallel computing because it operates without gradient information. The particles in the swarm can be updated independently, allowing for efficient utilization of parallel processing capabilities. This makes PSO well-suited for high-performance computing environments. Efficient global exploration: PSO is known for efficiently conducting global search space exploration. Collaboratively, the particles search the solution space, exchanging information and adjusting their positions in response to the best solutions so far discovered. This exploration feature improves the algorithm chances of locating the global optimum and prevents it from becoming stuck in local optima. Compared to genetic algorithms, PSO has advantages in terms of easy implementation, fewer parameters to adjust, and less computational complexity. Genetic algorithms typically require more genetic operators and involve higher computational costs due to operations like crossover and mutation. Therefore, in scenarios where simplicity, efficiency, and parallelization are important factors, PSO may be considered a favorable choice over genetic algorithms59. The PSO algorithm operates by iteratively updating the positions and velocities of a swarm of particles in the search space. Each particle represents a potential solution to the optimization problem, and its movement is influenced by its own best-known position (personal best) and the best-known position of the entire swarm (global best). The velocity and position of each particle are updated using the following equations:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V_{i} \left( {t + 1} \right) = wV_{i} \left( t \right) + c_{1} r_{1} \left( {P_{{{\text{best}},i}} - X_{i} \left( t \right)} \right) + c_{2} r_{2} \left( {G_{{{\text{best}}}} - X_{i} \left( t \right)} \right)$$\end{document}where: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${X}_{i}\left(t\right)$$\end{document} is the current position of particle \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i$$\end{document} at iteration \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document}.where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${X}_{i}\left(t+1\right)$$\end{document} is the new position of particle \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i$$\end{document} at iteration \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t + 1$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{i}\left(t\right)$$\end{document} is the velocity of particle \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i$$\end{document} at iteration \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w$$\end{document} ithe inertia weight, which controls the impact of the previous velocity on the current one. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c}_{1}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c}_{2}$$\end{document} are the cognitive and social coefficients, respectively, which determine the influence of the personal best and global best positions. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${r}_{1}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${r}_{2}$$\end{document} are random numbers uniformly distributed in the range [0, 1]. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{best,i}$$\end{document} is the best position found by particle \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i$$\end{document} so far. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${G}_{best}$$\end{document}is the best position found by any particle in the swarm so far. Algorithm 1WGAN-GP WGAN-GP 5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_{i} \left( {t + 1} \right) = X_{i} \left( t \right) + V_{i} \left( {t + 1} \right),$$\end{document} The inertia weight \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w$$\end{document} plays a crucial role in controlling the trade-off between exploration and exploitation. A higher value of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w$$\end{document} encourages exploration, while a lower value promotes exploitation. The cognitive and social coefficients \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c}_{1}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c}_{2}$$\end{document} determine the influence of the particle own experience and the swarm collective experience, respectively. The adoption of PSO in our NAS framework is motivated by several key properties of the architecture search space. First, PSO is inherently compatible with ordinal architectural parameters, such as the indices of layers or the number of filters, where preserving ordering constraints is crucial. Its smooth, momentum-based velocity–position update rules maintain these natural order relations, unlike abrupt mutations in genetic algorithms which may break them. Second, each candidate architecture in our method is represented as a fixed-length layer-index vector tailored to the search space, jointly encoding macro-level topology (e.g., layer sequence and connectivity) and micro-level configurations (e.g., operation types, kernel sizes, and pooling strategies). While some prior works use explicit two-level particle encoding, our representation implicitly covers both levels in a unified vector, enabling PSO to optimize them simultaneously. Finally, it should be emphasized that graph embeddings generated via graph2vec are employed exclusively for training the surrogate model, and PSO operates directly on the fixed-length layer-index vectors rather than on the graph embeddings. This clear distinction separates the embedding stage from the PSO-driven search process, preventing confusion about their respective roles, and highlights the suitability of PSO for exploring high-dimensional, non-convex architecture spaces60. Proposed method In this section, a comprehensive description of GNE-NAS (Neural Architecture Search, with GAN and Network Embedding) is provided, elucidating how Surrogate-Assisted Swarm Intelligence Algorithm (SASIA) is employed to explore the network embedding space for identifying the optimal architecture. As illustrated in Fig. 1, network embedding, surrogate models, and SI-based NAS are its three divisions.Fig. 1The overall structure of the proposed GNE-NAS framework. The overall structure of the proposed GNE-NAS framework. The first step is training the graph2vec model, which converts the architectures into embedding space. Then, a neural predictor is used as a surrogate model to guide the SI-based search by applying a GAN network for input data augmentation. In particular, we apply the infill criterion and robust environmental selection described in61. The framework of GNE-NAS is outlined in Algorithm 2. Algorithm 2GNE-NAS GNE-NAS This process starts with training a graph2vec model that maps architectures into the embedding space using an architecture dataset (line 1). Afterward, we set up a training dataset, (Arch), which includes N randomly generated architectures, and each architecture undergoes training. This training data serves as the initial population, denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{FE}$$\end{document}. Before utilizing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{FE}$$\end{document} to train the neural predictor surrogate model, we employ a GAN to generate a corresponding set of augmented data, denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{G}$$\end{document} (line 4). This augmentation process ensures a better representation for training neural predictor. The neural predictor surrogate model is then trained using the augmented and original training data (lines 5–6). Subsequently, until we reach the maximum number of real evaluations, we repeat the following steps (lines 8–26). During the surrogate-assisted evaluation phase in each generation, each individual’s real evaluation is replaced with the value predicted by neural predictor (lines 10–16). Using \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{FE}$$\end{document}, the initial population evaluation with surrogate assistance is established. After that, some individuals (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{infill}$$\end{document}) are selected using an infill criterion for actual assessments. In other words, the complete training of the network corresponding to each individual is conducted to acquire their validation accuracy (line 18). Following that, the individuals that have undergone real evaluations are gathered in Arch. Subsequently, the data within Arch are once again augmented by the GAN (line 21). The surrogate model is then updated (lines 22–23). Finally, the combined \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{FE}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{infill}$$\end{document} populations choose the candidate population for the following generation (lines 25–26). In the subsequent subsections, we provide detailed explanations of network embedding, surrogate model, particle swarm optimization algorithm, environmental selection, and infill sampling. Network embedding This refers to representing or embedding network structures in a lower-dimensional space. The goal is to capture the essential features or characteristics of the network in a more compact form. It`s a method or technique for embedding network structures using the skip-gram model, particularly for architecture-related applications. The skip-gram model is a type of word embedding model commonly used in NLP. It is part of the word2vec family of models and is designed to learn distributed representations of words in a continuous vector space. Regarding architecture, it seems to be adapted for network structures instead of words. Architecture embedding refers to the application of the embedding process to architectural structures. The embedded representations aim to capture the essential information about the network structure in a more manageable and interpretable way. We expect that when two architectures have similar or identical substructures, the vectors representing these substructures in the embedding space will be close to each other. In vector space, “closeness” implies similarity or relatedness. In our research, we consider rooted subgraphs as the substructures and a classifier is built for each of these rooted subgraphs. If we consider the vocabulary comprising all rooted subgraphs, the classifier output and weights become huge. T. Mikolov et al.55 suggest using the following formula for negative sampling:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log \sigma \left( {\Phi \left( G \right)w_{{sg_{j} }} } \right) + \mathop \sum \limits_{i = 1}^{k} {\mathbb{E}}_{{sg_{i} \sim P_{n} \left( {sg} \right)}} [\log \sigma ( - \Phi \left( G \right)w_{{sg_{i} }} )],$$\end{document} In the given equation, σ represents the sigmoid function, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{n}(sg)$$\end{document} denotes the noise distribution involving \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document} negative samples of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${sg}_{i}$$\end{document}. Each classifier aims to distinguish the target rooted subgraph \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${sg}_{i}$$\end{document} from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document} negative samples that are not part of the graph. Introduction to surrogate model A neural predictor is a neural network that ranks a set of items. This network estimates the probability distribution of ranks using generalized sigmoid functions. The advantages of using a neural predictor in problems such as NAS include the ability to rank items and replace accuracy calculations. Neural predictor provides the ability to rank neural architectures, which helps to determine the relative merits of different architectures. Instead of calculating the accuracy of each architecture, neural predictor can use the ranking score of the architectures, which is a different and more suitable criterion for evaluating different architectures in the neural architecture search environment. The procedure is as follows, when we apply two inputs to the neural predictor, it compares the score values of each input sample to determine which is higher. Then, using the sigmoid function, we calculate the likelihood that the first input will exceed the second. Finally, neural predictor then minimizes the subsequent cost function to train the base model:7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{ij} = P\left( {x_{i} > x_{j} } \right) = \frac{1}{{1 + e^{{ - \left( {s_{i} - s_{j} } \right)}} }}.$$\end{document}8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C = - \overline{P}_{ij} \log P_{ij} - \left( {1 - \overline{P}_{ij} } \right)\log \left( {1 - P_{ij} } \right)$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\overline{P}}_{ij}\in \left\{\text{0,1}\right\}$$\end{document} is target label. In Eq. (7) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{i}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{j}$$\end{document} are two embedding of architectures, each of which \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${s}_{i}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${s}_{j}$$\end{document} are attained scores from neural predictor. To increase the confidence level of surrogate model predictions, surrogate models need the output of uncertainty estimation of predictions. Since the neural predictor is regularly updated, on the other hand, in the skip-gram model, network embedding is accomplished via gradient descend; Thus, distinct embedding vectors are generated on each iteration, which are similar to each other. Therefore, by having several network embeddings and their predictions, it is possible to calculate the mean and standard deviation of the predictions. For better training of the surrogate model in the early stages, a GAN network is used to generate some individuals for training. The generated individuals are similar to the existing ones, ensuring that the network is trained with more data, leading to more accurate scores. Implementation detailed of the surrogate model The surrogate predictor adopts a RankNet-style feedforward neural network architecture. The input to the network is a 128-dimensional graph2vec embedding of an architecture, and the output is the predicted probability that the first architecture in a pair outperforms the second. The network consists of six fully-connected layers: 128 → 256 → 256 → 128 → 64 → 32 → 1 neurons, with ReLU6 activations and a final sigmoid output. Weights are initialized using Kaiming uniform initialization and biases set to zero. Training samples are created pairwise from architectures in the current population, labeled according to the sign of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F=100 - val\_loss$$\end{document}. The training set is shuffled before learning. The network is trained for 40 epochs with a batch size of 512, using the Adam optimizer (learning rate 0.001) and a StepLR scheduler (step size 15, gamma 0.5). Binary cross-entropy loss is minimized. All training is performed on GPU. The graph2vec embeddings are generated from the cell DAGs using a pretrained Doc2Vec model (dimension 128). The surrogate model is integrated in the ENAS search loop after the initial archive is populated with architectures from a random sampling of the NASBench search space. Uncertainty estimation To estimate the uncertainty associated with a candidate architecture, the network embedding procedure is repeated multiple times on the same graph representation with identical parameter settings. Although the input and parameters remain fixed, the stochasticity inherent in the embedding process produces slightly different vector representations in each repetition. These embeddings are evaluated individually by the surrogate performance predictor, and the standard deviation of the predicted performance values is reported as the uncertainty measure. Particle swarm optimization algorithm To avoid the completely random selection of individuals, the PSO algorithm is employed. The process begins with the random generation of particles, which are then updated in each generation using the PSO algorithm. In the early stages, the neural predictor network might not be fully trained, increasing the risk of incorrectly identifying the best architecture during the search process. Consequently, entering the evaluation phase prematurely could result in particles being prematurely excluded from the search process. To mitigate this, a secondary PSO algorithm is executed separately. With each global search, the value obtained is added to the evaluation list for the next stage. This secondary search process is then terminated to allow a new search for another optimal value. The values discovered are subsequently entered into the environmental selection phase, ensuring a more thorough and accurate evaluation. Architecture representation in search vs. surrogate phases In our framework, the PSO algorithm operates over a search-specific parameterization of architectures, where each particle’s position vector directly encodes the indices of operations and node connections in a cell. The continuous position update from Eqs. (4) and (5) is followed by a rounding and mapping step that deterministically yields a valid discrete neural architecture. In contrast, the 128-dimensional embeddings obtained via graph2vec are used exclusively for surrogate model training and prediction. These embeddings provide a continuous, learned representation of graph structure but are not decoded back to architectures during the search. This separation ensures that PSO search dynamics remain transparent and reproducible while the surrogate benefits from the rich structural information captured by embeddings. Environmental selection To simultaneously maintain high solution quality and promote diversity within the evolving population, we employ an environmental selection mechanism centered on a set of reference architectures. At each generation, all newly generated candidate architectures are first embedded and normalized, and the cosine distance between every offspring and each member of the current reference population is calculated. This yields a distance matrix that succinctly encodes structural similarities across the search space. Using this matrix, each offspring is assigned to the closest reference (i.e., the one with which it shares the minimum cosine distance), resulting in the formation of clusters, each anchored around a distinct reference. This assignment procedure is designed to ensure that candidates are distributed across different regions of the embedding space and that no single reference is overwhelmed by excessively many assignments. Within each cluster, only the most promising candidate, identified by its predicted fitness score, is selected to represent that region in the next generation’s population. This selection protocol is applied with further safeguards: depending on the current generation, either all reference members are forcibly replaced by their respective cluster’s best candidate, or replacement occurs only if the new candidate demonstrates higher predicted fitness. By repeating this process for every reference, we construct a new population that is both diverse (covering a range of architectural motifs) and comprised of strong performers. In this way, our approach not only minimizes the risk of premature convergence, but also prevents the loss of diversity caused by the population collapsing into a limited region of the search space or being overly attracted to dominant solutions61. Overall, this reference-based clustering and selection scheme provides an efficient, scalable method for preserving diversity without the need for explicit diversity metrics or post-hoc adjustments. Infill sampling To judiciously select candidate architectures for real evaluation, a two-objective criterion is adopted at each generation. Specifically, for every architecture in the candidate pool, the surrogate model provides a prediction of fitness \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{f}(x)$$\end{document}. Along with this, the model’s predictive uncertainty is estimated for each architecture, representing the degree of confidence in its prediction. To ensure balanced progress, candidates are prioritized using a non-dominated sorting approach on these two objectives: higher predicted fitness and higher uncertainty. An architecture is considered preferable if it either promises superior expected performance or if its evaluation could provide new information to further refine the surrogate model. The candidate pool is thus ranked into Pareto fronts, architectures in the leading fronts exhibit either high predictive scores or high uncertainty, or strike a balance between the two62. In implementation, architectures from the highest-ranked fronts are incrementally chosen until the desired number of samples (K, e.g., 10) is reached. If the final selected front includes more candidates than needed, its members are further ordered by predicted fitness, and those with highest predicted values are selected to fill the remaining slots. This method ensures that both exploration (sampling less certain regions of the search space) and exploitation (focusing on promising candidate solutions) are effectively maintained throughout the search. The chosen candidates subsequently undergo true evaluation, updating the surrogate for the next generation. Since this procedure is carried out after diversity-preserving environmental selection, the resulting sample set maintains architectural variety in every iteration. Experiments In this section, evaluations were conducted on three common search spaces from NASBench. The performance of the proposed method is examined on three NASBench search spaces and compared with existing NAS algorithms. All experiments were performed using an NVIDIA Geforce 1060 GTX GPU and Intel Core i7 8700 CPU. NASBench search space Three common search spaces in NASBench include NASBench101, NASBech201 and NASBench301. In the following, we will elaborate on these three search spaces in the NAS domain. NASBench-10163 is a benchmark for NAS, specifically designed to evaluate the effectiveness of various neural network topologies. It consists of 423,000 distinct convolutional architectures, each trained and evaluated three times with different random initializations on the CIFAR-10 dataset. Architectures are constructed by stacking cells, which form the search space. Each cell is represented as a directed acyclic graph with five nodes, where each node corresponds to an operation, and the adjacency matrix indicates the connections between these operations. To manage the complexity of the search space, only three types of operations are permitted: 3 × 3 convolution, 1 × 1 convolution, and 3 × 3 max-pooling. Additionally, the maximum number of edges in the adjacency matrix is limited to 9, further constraining the possible architectural configurations. NASBench-20164 serves as a comprehensive benchmark for NAS, comprising 15,625 unique architectures. Each architecture is built using four nodes and five possible operations: zeroize, skip-connect, 1 × 1 convolution, 3 × 3 convolution, and 3 × 3 average pooling. These architectures are represented graphically, where nodes correspond to feature maps and edges represent operations. NASBench-201 covers a wide variety of architectures, offering a diverse range of configurations for evaluation. It employs multiple datasets, including CIFAR-10, CIFAR-100, and ImageNet-16–120, to assess the architectures. For each architecture, test accuracy, validation, and training metrics are reported, providing a comprehensive performance assessment. NASBench-30165 is a surrogate benchmark tailored for large-scale neural architecture search, specifically extending to the DARTS search space. Unlike NASBench-101 and NASBench-201, which provide pre-computed performance metrics for all architectures, NASBench-301 employs a high-fidelity surrogate model trained on a subset of real evaluated architectures to predict performance for unseen ones. The search space consists of convolutional and reduction cells, each represented as a directed acyclic graph with seven nodes: two fixed input nodes, four intermediate nodes each with two incoming edges, and one output node that concatenates the outputs from the intermediate nodes. Edges represent operations selected from a predefined set, including separable convolutions (3 × 3, 5 × 5), dilated convolutions (3 × 3, 5 × 5), skip-connections, and zero-operations. The DARTS-like topology allows for a vast number of possible configurations, making exhaustive evaluation infeasible. NASBench-301 uses its surrogate predictor to provide fast accuracy estimates, enabling efficient benchmarking of NAS algorithms on large, continuous search spaces while maintaining a close approximation to real training results. Prediction analysis In this section, we analyze the performance of neural architecture predictors in NAS, focusing on how factors such as the search budget and training epochs influence prediction accuracy. By evaluating predictors across different search spaces, we aim to determine the optimal setup for balancing computational efficiency and accuracy. The following setup and results sections provide insights into this analysis. A.Setup. A.Setup. Setup. The time cost of NAS is directly impacted by the training epochs and search budget of the neural predictor. The search budget has a bigger impact on the NAS computational cost than the number of training epochs because it determines the size of the training dataset for the neural predictor. We examine how different search budgets affect the predictor in various search spaces to illustrate the performance of the neural predictor. The search budget ranges from 10 to 30 with a step size of 5, followed by increments of 10 up to 150. We also assess the correlation between the performance of the neural architecture as predicted by the neural predictor and its validation accuracy. For comparison, Kendall’s tau rank correlation is employed. B.Results. B.Results. Results. The predictive performance metrics of the neural predictor on NASBench-101 and NASBench-201 are shown in Figs. 2 and 3, respectively.Fig. 2Neural predictor performance prediction on NASBench-101.Fig. 3Neural predictor performance prediction on NASBench-201. (a) Kendall tau for cifar10. (b) Kendall tau for cifar100. (c) Kendall tau for ImageNet-16–120. Neural predictor performance prediction on NASBench-101. Neural predictor performance prediction on NASBench-201. (a) Kendall tau for cifar10. (b) Kendall tau for cifar100. (c) Kendall tau for ImageNet-16–120. According to Fig. 2 in the context of NASBench-101 search space, the performance of neural predictors improves as the search budget increases across different training epochs. Additionally, the performance of the neural predictor does not show a clear preference for any particular training epoch, although the 100-epoch training occasionally results in higher accuracy, despite some fluctuations. Therefore, we leverage a 50-epoch training budget for our neural predictor, especially when the search budget is small, because it provides a good starting point for neural architecture search. According to Fig. 3 in the NASBench-201 search space, neural architecture search was performed on three available databases in this search space: cifar10, cifar100, and ImageNet-16–120. As seen in Fig. 3a, b, and c, although there is no significant superiority among different training epochs with varying search budgets, all training epochs achieve high values of the Kendall Tau metric. Therefore, for both efficiency and acceptable performance reasons, we use 50 training epochs for training neural predictor tasks. This choice not only yields better results compared to other training epochs but also saves computational time and resources. Online learning for model training Online learning, known as incremental or streaming learning, plays a crucial role in training machine learning models. Unlike batch learning, where the entire dataset is processed at once, online learning updates the model incrementally as new data arrives. This approach is particularly useful when dealing with large datasets or dynamic environments. As you can see in Figs. 4 and 5, the effect of online learning on training neural predictor and searching for the best accuracy in two search spaces NasBench101 and NasBench201 is shown.Fig. 4Online learning on training neural predictor. Search in NasBench101_cifar10.Fig. 5Online learning on training neural predictor, Search in NasBench201 at different datasets (a) cifar10 (b) cifar100 (c) ImageNet-16–120. Online learning on training neural predictor. Search in NasBench101_cifar10. Online learning on training neural predictor, Search in NasBench201 at different datasets (a) cifar10 (b) cifar100 (c) ImageNet-16–120. The benefits of online learning include: Efficiency: Online learning allows models to adapt quickly to changing data, reducing the need for retraining from scratch. Scalability: It can handle continuous streams of data, making it suitable for real-time applications. Resources saving: By updating only on new data, computational resources are used more efficiently. Peer competitors The following describes the algorithms that we compared in our experiments:Random search serves as a straightforward yet effective baseline for NAS algorithms. The architecture with the highest validation accuracy is chosen as the final result after N randomly selected architectures are chosen from the search space.The Regularized Evolution Algorithm (REA) is an optimization approach for NAS that incorporates an age property within an EA. By assigning an age to each candidate solution, REA promotes diversity within the population, favoring younger individuals. This age-based regularization mechanism enhances exploration and prevents premature convergence, contributing to a more effective search for optimal neural architectures.BANANAS, one NAS algorithm that uses a path-based encoding scheme to represent architectures is called BANANAS (Bayesian Optimization Algorithm for Neural Architecture Search). As a surrogate model, the algorithm uses an ensemble of feedforward neural networks that have been trained to forecast architecture performance using their encoded representations. By employing Bayesian optimization, BANANAS efficiently explores the complex search space of neural architectures, aiming to reduce the computational cost associated to evaluating each architecture by making informed predictions through its surrogate model. This approach accelerates the NAS process by guiding the search toward promising candidate architectures.GP-bayesopt, a NAS algorithm from33, uses the Upper Confidence Bound (UCB) acquisition function in conjunction with a Gaussian process model and Bayesian optimization. The algorithm uses a probabilistic model to illustrate how different neural network architectures perform, adapting its search strategy through iterative evaluations. By balancing exploration and exploitation, GP-bayesopt efficiently navigates the architecture search space, aiming to discover architectures with improved performance.Deep Networks for Global Optimization (DNGO)66 is a Bayesian optimization algorithm that uses basis functions taken out of neural networks. The algorithm employs a probabilistic model, utilizing a neural network to capture complex patterns in the objective function. This allows DNGO to adaptively model the underlying structure of the optimization problem, facilitating effective decision-making in the search for the global optimum.Bohamiann is a Bayesian optimization algorithm that incorporates Hamiltonian Monte Carlo and artificial neural networks33. The algorithm utilizes a Bayesian neural network as its surrogate model, leveraging the power of Hamiltonian Monte Carlo to efficiently explore the search space. This approach allows Bohamiann to make informed decisions during optimization, leading to effective and robust performance in finding optimal solutions.GCN-predictor33 employs Bayesian optimization as its optimization strategy and utilizes GCNs as the surrogate model. This approach leverages the power of graph-based representations to model the architecture space efficiently. The Bayesian optimization framework guides the exploration of architectures, making it a robust method for optimizing neural architecture search problems.BONAS67 is a Bayesian optimization algorithm that utilizes a GCN as its surrogate model. The primary objective of BONAS is to discover optimal neural network architectures. To achieve this, it incorporates a weighted loss function that emphasizes architectures with high performance during the optimization process. This weighted loss helps guide the search towards architectures that are more likely to yield superior results.Arch2vec-RL and Arch2vec-BO9 utilize various optimization techniques, such as Bayesian and reinforcement learning, to find the best neural network architecture. These methods also incorporate the arch2vec embedding technique, which uses an autoencoder that varies to learn architecture representations. The goal is to enhance the efficiency and effectiveness of the architecture search process by combining these optimization algorithms with the learned architecture embeddings.NPENAS-SSRL and NPENAS-SSCCL11 use two self-supervised learning models to pre-train the architecture embeddings and evolutionary algorithms as optimizer. Predicting the distance between architectures is a pretext task that the first model uses to learn the embeddings. To further improve the learning of architecture embeddings, the second model builds positive and negative samples and presents a contrastive learning algorithm. During the optimization process, these methods seek to enhance the representation of neural network architectures by utilizing self-supervised learning techniques.SAENAS-NE61 proposes a novel neural architecture search method. It combines Surrogate-Assisted Evolutionary Algorithm (SAEA) with a Network Embedding (NE) approach. Key contributions include efficient architecture sampling, and improved performance through network embedding. Random search serves as a straightforward yet effective baseline for NAS algorithms. The architecture with the highest validation accuracy is chosen as the final result after N randomly selected architectures are chosen from the search space. Random search serves as a straightforward yet effective baseline for NAS algorithms. The architecture with the highest validation accuracy is chosen as the final result after N randomly selected architectures are chosen from the search space. The Regularized Evolution Algorithm (REA) is an optimization approach for NAS that incorporates an age property within an EA. By assigning an age to each candidate solution, REA promotes diversity within the population, favoring younger individuals. This age-based regularization mechanism enhances exploration and prevents premature convergence, contributing to a more effective search for optimal neural architectures. The Regularized Evolution Algorithm (REA) is an optimization approach for NAS that incorporates an age property within an EA. By assigning an age to each candidate solution, REA promotes diversity within the population, favoring younger individuals. This age-based regularization mechanism enhances exploration and prevents premature convergence, contributing to a more effective search for optimal neural architectures. BANANAS, one NAS algorithm that uses a path-based encoding scheme to represent architectures is called BANANAS (Bayesian Optimization Algorithm for Neural Architecture Search). As a surrogate model, the algorithm uses an ensemble of feedforward neural networks that have been trained to forecast architecture performance using their encoded representations. By employing Bayesian optimization, BANANAS efficiently explores the complex search space of neural architectures, aiming to reduce the computational cost associated to evaluating each architecture by making informed predictions through its surrogate model. This approach accelerates the NAS process by guiding the search toward promising candidate architectures. BANANAS, one NAS algorithm that uses a path-based encoding scheme to represent architectures is called BANANAS (Bayesian Optimization Algorithm for Neural Architecture Search). As a surrogate model, the algorithm uses an ensemble of feedforward neural networks that have been trained to forecast architecture performance using their encoded representations. By employing Bayesian optimization, BANANAS efficiently explores the complex search space of neural architectures, aiming to reduce the computational cost associated to evaluating each architecture by making informed predictions through its surrogate model. This approach accelerates the NAS process by guiding the search toward promising candidate architectures. GP-bayesopt, a NAS algorithm from33, uses the Upper Confidence Bound (UCB) acquisition function in conjunction with a Gaussian process model and Bayesian optimization. The algorithm uses a probabilistic model to illustrate how different neural network architectures perform, adapting its search strategy through iterative evaluations. By balancing exploration and exploitation, GP-bayesopt efficiently navigates the architecture search space, aiming to discover architectures with improved performance. GP-bayesopt, a NAS algorithm from33, uses the Upper Confidence Bound (UCB) acquisition function in conjunction with a Gaussian process model and Bayesian optimization. The algorithm uses a probabilistic model to illustrate how different neural network architectures perform, adapting its search strategy through iterative evaluations. By balancing exploration and exploitation, GP-bayesopt efficiently navigates the architecture search space, aiming to discover architectures with improved performance. Deep Networks for Global Optimization (DNGO)66 is a Bayesian optimization algorithm that uses basis functions taken out of neural networks. The algorithm employs a probabilistic model, utilizing a neural network to capture complex patterns in the objective function. This allows DNGO to adaptively model the underlying structure of the optimization problem, facilitating effective decision-making in the search for the global optimum. Deep Networks for Global Optimization (DNGO)66 is a Bayesian optimization algorithm that uses basis functions taken out of neural networks. The algorithm employs a probabilistic model, utilizing a neural network to capture complex patterns in the objective function. This allows DNGO to adaptively model the underlying structure of the optimization problem, facilitating effective decision-making in the search for the global optimum. Bohamiann is a Bayesian optimization algorithm that incorporates Hamiltonian Monte Carlo and artificial neural networks33. The algorithm utilizes a Bayesian neural network as its surrogate model, leveraging the power of Hamiltonian Monte Carlo to efficiently explore the search space. This approach allows Bohamiann to make informed decisions during optimization, leading to effective and robust performance in finding optimal solutions. Bohamiann is a Bayesian optimization algorithm that incorporates Hamiltonian Monte Carlo and artificial neural networks33. The algorithm utilizes a Bayesian neural network as its surrogate model, leveraging the power of Hamiltonian Monte Carlo to efficiently explore the search space. This approach allows Bohamiann to make informed decisions during optimization, leading to effective and robust performance in finding optimal solutions. GCN-predictor33 employs Bayesian optimization as its optimization strategy and utilizes GCNs as the surrogate model. This approach leverages the power of graph-based representations to model the architecture space efficiently. The Bayesian optimization framework guides the exploration of architectures, making it a robust method for optimizing neural architecture search problems. GCN-predictor33 employs Bayesian optimization as its optimization strategy and utilizes GCNs as the surrogate model. This approach leverages the power of graph-based representations to model the architecture space efficiently. The Bayesian optimization framework guides the exploration of architectures, making it a robust method for optimizing neural architecture search problems. BONAS67 is a Bayesian optimization algorithm that utilizes a GCN as its surrogate model. The primary objective of BONAS is to discover optimal neural network architectures. To achieve this, it incorporates a weighted loss function that emphasizes architectures with high performance during the optimization process. This weighted loss helps guide the search towards architectures that are more likely to yield superior results. BONAS67 is a Bayesian optimization algorithm that utilizes a GCN as its surrogate model. The primary objective of BONAS is to discover optimal neural network architectures. To achieve this, it incorporates a weighted loss function that emphasizes architectures with high performance during the optimization process. This weighted loss helps guide the search towards architectures that are more likely to yield superior results. Arch2vec-RL and Arch2vec-BO9 utilize various optimization techniques, such as Bayesian and reinforcement learning, to find the best neural network architecture. These methods also incorporate the arch2vec embedding technique, which uses an autoencoder that varies to learn architecture representations. The goal is to enhance the efficiency and effectiveness of the architecture search process by combining these optimization algorithms with the learned architecture embeddings. Arch2vec-RL and Arch2vec-BO9 utilize various optimization techniques, such as Bayesian and reinforcement learning, to find the best neural network architecture. These methods also incorporate the arch2vec embedding technique, which uses an autoencoder that varies to learn architecture representations. The goal is to enhance the efficiency and effectiveness of the architecture search process by combining these optimization algorithms with the learned architecture embeddings. NPENAS-SSRL and NPENAS-SSCCL11 use two self-supervised learning models to pre-train the architecture embeddings and evolutionary algorithms as optimizer. Predicting the distance between architectures is a pretext task that the first model uses to learn the embeddings. To further improve the learning of architecture embeddings, the second model builds positive and negative samples and presents a contrastive learning algorithm. During the optimization process, these methods seek to enhance the representation of neural network architectures by utilizing self-supervised learning techniques. NPENAS-SSRL and NPENAS-SSCCL11 use two self-supervised learning models to pre-train the architecture embeddings and evolutionary algorithms as optimizer. Predicting the distance between architectures is a pretext task that the first model uses to learn the embeddings. To further improve the learning of architecture embeddings, the second model builds positive and negative samples and presents a contrastive learning algorithm. During the optimization process, these methods seek to enhance the representation of neural network architectures by utilizing self-supervised learning techniques. SAENAS-NE61 proposes a novel neural architecture search method. It combines Surrogate-Assisted Evolutionary Algorithm (SAEA) with a Network Embedding (NE) approach. Key contributions include efficient architecture sampling, and improved performance through network embedding. SAENAS-NE61 proposes a novel neural architecture search method. It combines Surrogate-Assisted Evolutionary Algorithm (SAEA) with a Network Embedding (NE) approach. Key contributions include efficient architecture sampling, and improved performance through network embedding. Parameter settings All algorithms are implemented with the same set of parameter configurations, ensuring uniformity across experiments, and these settings are carefully aligned with the detailed specifications and guidelines provided in references11,33.The dimensionality of the architecture embeddings is fixed at 128 for NASBench-101, NASBench-201 and NASBench-301.For NASBench-101, NASBench-201 and NASBench-301, the maximum number of real evaluations is 150, 100 and 300, respectively. The dimensionality of the architecture embeddings is fixed at 128 for NASBench-101, NASBench-201 and NASBench-301. The dimensionality of the architecture embeddings is fixed at 128 for NASBench-101, NASBench-201 and NASBench-301. For NASBench-101, NASBench-201 and NASBench-301, the maximum number of real evaluations is 150, 100 and 300, respectively. For NASBench-101, NASBench-201 and NASBench-301, the maximum number of real evaluations is 150, 100 and 300, respectively. The detailed parameter settings for GNE-NAS are as follows:For rooted subgraphs, the maximum degree that can be allowed is 2. Within the designated search space, a subgraph with the degree of 2 can satisfactorily represent up to 5 network layers within a cell.The cosine learning rate schedule is used, with the initial learning rate set to 0.025.Effective training of the graph2vec model is ensured by using 60 epochs for training.For the first thirty real evaluations, the number of new architectures (K) chosen from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{{t}_{max}}$$\end{document} is set to 5, and after that, set to 10.For both NASBench-101 and NASBench-201, the initial population size is set to 10 and for NASbench-301 is set to 30. For rooted subgraphs, the maximum degree that can be allowed is 2. Within the designated search space, a subgraph with the degree of 2 can satisfactorily represent up to 5 network layers within a cell. For rooted subgraphs, the maximum degree that can be allowed is 2. Within the designated search space, a subgraph with the degree of 2 can satisfactorily represent up to 5 network layers within a cell. The cosine learning rate schedule is used, with the initial learning rate set to 0.025. The cosine learning rate schedule is used, with the initial learning rate set to 0.025. Effective training of the graph2vec model is ensured by using 60 epochs for training. Effective training of the graph2vec model is ensured by using 60 epochs for training. For the first thirty real evaluations, the number of new architectures (K) chosen from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{{t}_{max}}$$\end{document} is set to 5, and after that, set to 10. For the first thirty real evaluations, the number of new architectures (K) chosen from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{{t}_{max}}$$\end{document} is set to 5, and after that, set to 10. For both NASBench-101 and NASBench-201, the initial population size is set to 10 and for NASbench-301 is set to 30. For both NASBench-101 and NASBench-201, the initial population size is set to 10 and for NASbench-301 is set to 30. Neural architecture search performance To evaluate the performance of GNE-NAS, we compare its results with those of other algorithms discussed in the peer competitor section, using NASBench-101, NASBench-201 and NASBench-301 as benchmarks. NASBench-101 results Based on 200 independent runs on NASBench-101, Table 1 presents the performance comparison of various NAS algorithms. To determine the accuracy of each candidate architecture, the top two algorithms, Random search, and REA train the network from scratch using real evaluations. Using the original encoding of the architectures, the middle six algorithms, BANANAS, GP-bayesopt, DNGO, Bohamiann, GCN-Predictor, and BONAS, create surrogate models. The bottom six algorithms, Arch2vec-RL, Arch2vec-BO, NPENAS-SSRL, NPENAS-CCL, SAENAS-NE, and GNE-NAS, use unsupervised or self-supervised algorithms to map the architectures into the embedding space.Table 1Comparison of 13 Algorithms with GNE-NAS on NASBench-101.MethodsSearch budgetValidationArchitecture embeddingRandom search15094.33 ± 0.12–REA15094.40 ± 0.18–BANANAS15094.65 ± 0.14SupervisedGP-bayesopt15094.67 ± 0.15SupervisedDNGO15094.54 ± 0.16SupervisedBohamiann15094.53 ± 0.17SupervisedGCN-Predictor15094.33 ± 0.11SupervisedBONAS15094.63 ± 0.12SupervisedArch2vec-RL15094.35 ± 0.14UnsupervisedArch2vec-BO15094.44 ± 0.25UnsupervisedNPENAS-SSRL15094.76 ± 0.19Self-supervisedNPENAS-CCL15094.72 ± 0.17Self-supervisedSAENAS-NE15094.72 ± 0.14UnsupervisedGNE-NAS15094.78 ± 0.19UnsupervisedBold values indicate the best performance among all compared methods for each metric. Table 1Comparison of 13 Algorithms with GNE-NAS on NASBench-101.MethodsSearch budgetValidationArchitecture embeddingRandom search15094.33 ± 0.12–REA15094.40 ± 0.18–BANANAS15094.65 ± 0.14SupervisedGP-bayesopt15094.67 ± 0.15SupervisedDNGO15094.54 ± 0.16SupervisedBohamiann15094.53 ± 0.17SupervisedGCN-Predictor15094.33 ± 0.11SupervisedBONAS15094.63 ± 0.12SupervisedArch2vec-RL15094.35 ± 0.14UnsupervisedArch2vec-BO15094.44 ± 0.25UnsupervisedNPENAS-SSRL15094.76 ± 0.19Self-supervisedNPENAS-CCL15094.72 ± 0.17Self-supervisedSAENAS-NE15094.72 ± 0.14UnsupervisedGNE-NAS15094.78 ± 0.19UnsupervisedBold values indicate the best performance among all compared methods for each metric. Comparison of 13 Algorithms with GNE-NAS on NASBench-101. Bold values indicate the best performance among all compared methods for each metric. Our proposed method, GNE-NAS, demonstrates better performance on NASBench-101. According to Table 1, our proposed method, GNE-NAS, achieves a validation accuracy of 94.78, which is not only the highest among all the compared algorithms but also demonstrates the significant superiority of this method in finding optimal architectures. This result indicates that GNE-NAS has been able to identify high-performance architectures by utilizing embedded mapping mechanisms. Furthermore, the standard deviation of the proposed method is 0.19, reflecting the high stability and reliability of this method across different runs. This value, compared to other algorithms with standard deviations ranging from 0.11 (minimum) to 0.25 (maximum), falls within a very acceptable range. This stability shows that GNE-NAS not only achieves better results but also provides consistent and reproducible outcomes in repeated executions. To better understand these results, it can be noted that methods based on surrogate models and unsupervised learning, such as GNE-NAS, demonstrate higher efficiency in exploring the architecture search space by reducing reliance on costly and time-consuming evaluations. This feature is particularly valuable in large and complex datasets like NASBench-101. Finally, the superior performance of GNE-NAS can be attributed to the intelligent use of embedding algorithms, surrogate model and PSO. By focusing on more promising regions, this method has been able to achieve more accurate and stable results. NASBench-201 results Table 2 presents the results of 13 other algorithms and 200 independent runs of GNE-NAS on NASBench-201. In these experiments, the performance of GNE-NAS was evaluated on three datasets, CIFAR-10, CIFAR-100, and ImageNet-16–120. The results demonstrate that GNE-NAS outperforms other algorithms in terms of validation accuracy. Specifically, for ImageNet-16–120, GNE-NAS achieved the highest validation accuracy compared to some other methods.Table 2The performance of the 12 compared algorithms and GNE-NAS on NASBench-201.Cifar10Cifar100ImageNet-16–120MethodsSearch budgetValidationTestValidationTestValidationTestRandom search10091.07 ± 0.2793.82 ± 0.2471.44 ± 0.8371.40 ± 0.8345.37 ± 0.6345.36 ± 0.67REA10091.37 ± 0.2594.06 ± 0.2972.79 ± 0.6972.72 ± 0.7246.15 ± 0.4545.99 ± 0.51BANANAS10091.50 ± 0.1594.23 ± 0.3073.27 ± 0.5773.25 ± 0.6346.45 ± 0.2546.31 ± 0.31GP-bayesopt10091.45 ± 0.2394.16 ± 0.3173.11 ± 0.6573.05 ± 0.7546.51 ± 0.2546.25 ± 0.34DNGO10091.41 ± 0.1794.08 ± 0.2672.71 ± 0.6672.66 ± 0.6746.11 ± 0.4446.00 ± 0.48Bohamiann10091.41 ± 0.1894.09 ± 0.2672. 73 ± 0.6472.65 ± 0.6646.15 ± 0.4546.03 ± 0.48GCN-Predictor10091.02 ± 0.3293.74 ± 0.3371.43 ± 0.7271.43 ± 0.7745.47 ± 0.7245.36 ± 0.78BONAS10091.56 ± 0.1094.32 ± 0.1573.32 ± 0.4073.30 ± 0.4646.56 ± 0.1946.31 ± 0.30Arch2vec-RL10091.43 ± 0.2894.23 ± 0.2673.12 ± 0.6673.06 ± 0.8746.32 ± 0.2046.35 ± 0.35Arch2vec-BO10091.51 ± 0.1394.31 ± 0.1473.38 ± 0.3473.44 ± 0.2246.33 ± 0.2146.35 ± 0.27NPENAS-SSRL10091.56 ± 0.1494.32 ± 0.1973.47 ± 0.2273.47 ± 0.3046.53 ± 0.3345.83 ± 0.60NPENAS-CCL10091.57 ± 0.1394.32 ± 0.1973.48 ± 0.1573.49 ± 0.2346.62 ± 0.3445.61 ± 0.41SAENAS-NE10091.58 ± 0.0994.34 ± 0.1273.46 ± 0.1573.46 ± 0.2046.59 ± 0.1446.36 ± 0.26GNE-NAS10091.58 ± 0.0594.34 ± 0.0973.46 ± 0.0773.44 ± 0.1246.63 ± 0.1346.30 ± 0.25Bold values indicate the best performance among all compared methods for each metric. Table 2The performance of the 12 compared algorithms and GNE-NAS on NASBench-201.Cifar10Cifar100ImageNet-16–120MethodsSearch budgetValidationTestValidationTestValidationTestRandom search10091.07 ± 0.2793.82 ± 0.2471.44 ± 0.8371.40 ± 0.8345.37 ± 0.6345.36 ± 0.67REA10091.37 ± 0.2594.06 ± 0.2972.79 ± 0.6972.72 ± 0.7246.15 ± 0.4545.99 ± 0.51BANANAS10091.50 ± 0.1594.23 ± 0.3073.27 ± 0.5773.25 ± 0.6346.45 ± 0.2546.31 ± 0.31GP-bayesopt10091.45 ± 0.2394.16 ± 0.3173.11 ± 0.6573.05 ± 0.7546.51 ± 0.2546.25 ± 0.34DNGO10091.41 ± 0.1794.08 ± 0.2672.71 ± 0.6672.66 ± 0.6746.11 ± 0.4446.00 ± 0.48Bohamiann10091.41 ± 0.1894.09 ± 0.2672. 73 ± 0.6472.65 ± 0.6646.15 ± 0.4546.03 ± 0.48GCN-Predictor10091.02 ± 0.3293.74 ± 0.3371.43 ± 0.7271.43 ± 0.7745.47 ± 0.7245.36 ± 0.78BONAS10091.56 ± 0.1094.32 ± 0.1573.32 ± 0.4073.30 ± 0.4646.56 ± 0.1946.31 ± 0.30Arch2vec-RL10091.43 ± 0.2894.23 ± 0.2673.12 ± 0.6673.06 ± 0.8746.32 ± 0.2046.35 ± 0.35Arch2vec-BO10091.51 ± 0.1394.31 ± 0.1473.38 ± 0.3473.44 ± 0.2246.33 ± 0.2146.35 ± 0.27NPENAS-SSRL10091.56 ± 0.1494.32 ± 0.1973.47 ± 0.2273.47 ± 0.3046.53 ± 0.3345.83 ± 0.60NPENAS-CCL10091.57 ± 0.1394.32 ± 0.1973.48 ± 0.1573.49 ± 0.2346.62 ± 0.3445.61 ± 0.41SAENAS-NE10091.58 ± 0.0994.34 ± 0.1273.46 ± 0.1573.46 ± 0.2046.59 ± 0.1446.36 ± 0.26GNE-NAS10091.58 ± 0.0594.34 ± 0.0973.46 ± 0.0773.44 ± 0.1246.63 ± 0.1346.30 ± 0.25Bold values indicate the best performance among all compared methods for each metric. The performance of the 12 compared algorithms and GNE-NAS on NASBench-201. Bold values indicate the best performance among all compared methods for each metric. However, for the CIFAR-100 dataset, slight differences in performance were observed between GNE-NAS and NPENAS-CCL. Specifically, the differences in validation accuracy and test accuracy between GNE-NAS and NPENAS-CCL were 0.02 and 0.05, respectively, indicating that GNE-NAS performed slightly weaker in these cases. Nevertheless, these differences are minimal and do not undermine the overall superiority of the proposed method. Another noteworthy point is the standard deviation values obtained for validation accuracy and test accuracy, which were 0.07 and 0.12, respectively. These values are significantly lower than the corresponding values in other methods. This indicates the high stability and reliability of the proposed GNE-NAS method within the NASBench-201 framework. High stability means that GNE-NAS not only achieves good results but also provides consistent and reproducible outcomes in repeated runs. Additionally, the use of advanced optimization algorithms in GNE-NAS has enabled this method to deliver competitive and, in some cases, superior performance across various datasets, including CIFAR-10, CIFAR-100, and ImageNet-16–120. In conclusion, it can be inferred that GNE-NAS not only excels in terms of validation and test accuracy but also offers significant advantages in stability and reliability compared to other methods. These features establish GNE-NAS as a powerful and reliable approach in neural architecture search. NASBench-301 results Table 3 presents the results of GNE-NAS on the NASBench-301 benchmark, which serves as a large-scale and DARTS-like architecture search space. NASBench-301 significantly differs from NASBench-101 and NASBench-201, as it encompasses an extremely large and diverse search space, estimated to contain approximately 1018 valid architectures, where the performance of each architecture has been estimated using a surrogate model trained on extensive real evaluations.Table 3The performance of the 13 compared algorithms and GNE-NAS on NASBench-301.MethodSearch budgetValidationArchitecture embeddingRandom search30094.34 ± 0.1–REA30094.86 ± 0.11–BANANAS30094.89 ± 0.11SupervisedGp_bayesopt30094.98 ± 0.11SupervisedDNGO30094.85 ± 0.10SupervisedBohamian30094.85 ± 0.10SupervisedGCN-Predictor30094.35 ± 0.11SupervisedBONAS30094.91 ± 0.10SupervisedArch2vec-RL30094.48 ± 0.08UnsupervisedArch2vec-BO30094.43 ± 0.12UnsupervisedNPENAS-CCL30094.92 ± 0.12Self-supervisedSAENAS-NE30095.01 ± 0.07UnsupervisedGNE-NAS30095.02 ± 0.11UnsupervisedBold values indicate the best performance among all compared methods for each metric. Table 3The performance of the 13 compared algorithms and GNE-NAS on NASBench-301.MethodSearch budgetValidationArchitecture embeddingRandom search30094.34 ± 0.1–REA30094.86 ± 0.11–BANANAS30094.89 ± 0.11SupervisedGp_bayesopt30094.98 ± 0.11SupervisedDNGO30094.85 ± 0.10SupervisedBohamian30094.85 ± 0.10SupervisedGCN-Predictor30094.35 ± 0.11SupervisedBONAS30094.91 ± 0.10SupervisedArch2vec-RL30094.48 ± 0.08UnsupervisedArch2vec-BO30094.43 ± 0.12UnsupervisedNPENAS-CCL30094.92 ± 0.12Self-supervisedSAENAS-NE30095.01 ± 0.07UnsupervisedGNE-NAS30095.02 ± 0.11UnsupervisedBold values indicate the best performance among all compared methods for each metric. The performance of the 13 compared algorithms and GNE-NAS on NASBench-301. Bold values indicate the best performance among all compared methods for each metric. Despite this vast complexity and openness, as shown in Table 3, GNE-NAS achieves higher accuracy compared to competing methods on NASBench-301, albeit with a slightly higher variance across runs. Nevertheless, the observed standard deviation remains within an acceptable range, indicating that the method retains a reasonable level of stability even in such a challenging and high-dimensional search space. The ability of GNE-NAS to maintain high accuracy and low variance on NASBench-301 demonstrates its strong generalizability and robustness beyond fixed-cell and small-scale NAS benchmarks such as NASBench-101 and NASBench-201. This is particularly noteworthy, as NASBench-301 is widely regarded as one of the closest available proxies to real-world, large-scale NAS scenarios. Overall, these results indicate that GNE-NAS is not limited to cellular or small search spaces, and can be effectively deployed in much broader and more complex architecture search problems. The findings confirm that the design choices in GNE-NAS, including its embedding mechanism, surrogate modeling, and population-based search strategy, are scalable and versatile for diverse NAS applications. Top-performing neural architectures Table 4 presents the best architectures and accuracies obtained in each search space for different datasets.Table 4Best architecture and accuracy in each search space for different datasets.Search spaceDatasetBest architectureBest accuracyNasbench101Cifar10{‘matrix’: array ([[0, 1, 1, 1, 1, 1, 1],[0, 0, 0, 1, 0, 1, 0],[0, 0, 0, 0, 1, 1, 0],[0, 0, 0, 0, 1, 0, 0],[0, 0, 0, 0, 0, 0, 1],[0, 0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0, 0]]),‘ops’: [‘input’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘output’]}95.05NasBench201Cifar10{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_3 × 3 ~ 0|nor_conv_3 × 3 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_1 × 1 ~ 2|’}91.61NasBench201Cifar100{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_3 × 3 ~ 0|nor_conv_3 × 3 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_3 × 3 ~ 2|’}73.49NasBench201ImageNet16_120{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_1 × 1 ~ 0|nor_conv_1 × 1 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_3 × 3 ~ 2|’}46.77 Table 4Best architecture and accuracy in each search space for different datasets.Search spaceDatasetBest architectureBest accuracyNasbench101Cifar10{‘matrix’: array ([[0, 1, 1, 1, 1, 1, 1],[0, 0, 0, 1, 0, 1, 0],[0, 0, 0, 0, 1, 1, 0],[0, 0, 0, 0, 1, 0, 0],[0, 0, 0, 0, 0, 0, 1],[0, 0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0, 0]]),‘ops’: [‘input’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘output’]}95.05NasBench201Cifar10{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_3 × 3 ~ 0|nor_conv_3 × 3 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_1 × 1 ~ 2|’}91.61NasBench201Cifar100{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_3 × 3 ~ 0|nor_conv_3 × 3 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_3 × 3 ~ 2|’}73.49NasBench201ImageNet16_120{‘arch’: ‘|nor_conv_3 × 3 ~ 0| +|nor_conv_1 × 1 ~ 0|nor_conv_1 × 1 ~ 1| +|skip_connect ~ 0|nor_conv_3 × 3 ~ 1|nor_conv_3 × 3 ~ 2|’}46.77 Best architecture and accuracy in each search space for different datasets. {‘matrix’: array ([[0, 1, 1, 1, 1, 1, 1], [0, 0, 0, 1, 0, 1, 0], [0, 0, 0, 0, 1, 1, 0], [0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 0, 1], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0]]), ‘ops’: [‘input’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘conv3 × 3-bn-relu’, ‘output’]} Top Architecture from NAS-Bench-101: In the NAS-Bench-101 search space, the top-performing architecture for the CIFAR-10 dataset is optimized using a connection matrix and operations such as 3 × 3 convolution. This architecture, achieving a high accuracy of 95.05%, effectively extracts key image features through successive convolutional layers 3 × 3 and 1 × 1 combined with batch normalization and activation functions, such as ReLU, and pooling operations, such as maxpool3 × 3. Top Architectures from NAS-Bench-201: For NAS-Bench-201, three datasets, CIFAR-10, CIFAR-100, and ImageNet-16–120, were examined. The optimized architectures in this space consist of a combination of 3 × 3 and 1 × 1 convolutions along with skip connections. Depending on the complexity and size of the dataset, these architectures achieved varying accuracies, ranging from 91.61% to 46.77%. For instance, in the case of the CIFAR-100 dataset, the architectural combination of multiple 3 × 3 convolutions and skip connections yielded an accuracy of 73.49%. These results clearly demonstrate that selecting the best architecture for each dataset requires a deep understanding of the data’s characteristics and the model’s complexities. The optimized architectures in NAS-Bench-201 have shown superior performance across various datasets, particularly due to their flexibility in combining different layers and utilizing skip connections. This is especially evident in more complex datasets like ImageNet-16–120. Ablation study Network embedding methods vs. direct NAS-bench architecture coding To comprehensively evaluate the contribution of network embedding to search performance, we compare multiple embedding methods (graph2vec, GCN-based embedding, and transformer-based embedding) with a baseline that uses the raw NAS-Bench architecture coding without any embedding. All experiments are conducted under identical search settings to ensure a fair comparison. Five diverse benchmarks were used for this comparison:NASBench-101 (CIFAR-10)NASBench-201 (CIFAR-10, CIFAR-100, ImageNet16-120)NASBench-301 (CIFAR-10) NASBench-101 (CIFAR-10) NASBench-101 (CIFAR-10) NASBench-201 (CIFAR-10, CIFAR-100, ImageNet16-120) NASBench-201 (CIFAR-10, CIFAR-100, ImageNet16-120) NASBench-301 (CIFAR-10) NASBench-301 (CIFAR-10) The results for each are summarized in Fig. 6.Fig. 6Evaluate the effect of embedding methods with different search budget. Evaluate the effect of embedding methods with different search budget. Comparative findings NASBench-101 (CIFAR-10). NASBench-101 (CIFAR-10). NASBench-101 (CIFAR-10). All three embeddings (graph2vec, GCN, transformer) achieve very similar performance, with overlapping accuracy curves. In contrast, the Raw baseline (green curve) lags behind in accuracy across all search budgets and shows slightly higher variance, especially in the early phase. This suggests that embedding methods provide a tangible advantage in this benchmark. 2.NASBench-201 (CIFAR-10). 2.NASBench-201 (CIFAR-10). NASBench-201 (CIFAR-10). Again, all three embeddings yield comparable accuracy, with only minor fluctuations due to randomness. The Raw baseline remains clearly below the embedding-based methods for the entire search, indicating that omitting an embedding reduces surrogate performance in this setting. 3.NASBench-201 (CIFAR-100). 3.NASBench-201 (CIFAR-100). NASBench-201 (CIFAR-100). On this more challenging dataset, the performance gap widens. While GCN, transformer, and graph2vec converge to similar high accuracy, the Raw baseline starts much lower and improves more slowly, with significantly larger variance bars. This highlights that embeddings become more beneficial as task difficulty increases. 4.NASBench-201 (ImageNet16-120). 4.NASBench-201 (ImageNet16-120). NASBench-201 (ImageNet16-120). Embedding-based methods reach competitive top accuracy, with graph2vec showing slightly more consistent stability. The Raw baseline is well below these in early and mid-phases and only approaches them toward the end, while maintaining the highest variance. 5.NASBench-301 (CIFAR-10). 5.NASBench-301 (CIFAR-10). NASBench-301 (CIFAR-10). In this DARTS-like search space, graph2vec, GCN, and transformer again show highly similar trajectories with overlapping curves. The Raw baseline starts with noticeably lower accuracy but gradually narrows the gap; in some mid-to-late stages, it even matches or slightly surpasses the transformer curve. Nonetheless, embeddings generally reach high-accuracy regions faster, indicating an efficiency advantage when search budget is limited. Across the five benchmarks, NASBench-101 (CIFAR-10), NASBench-201 (CIFAR-10, CIFAR-100, ImageNet16-120), and NASBench-301 (CIFAR-10), embedding-based surrogates generally deliver higher accuracy than the Raw baseline, with differences ranging from small (NASBench-101 CIFAR-10) to substantial (NASBench-201 CIFAR-100 and ImageNet16-120). In NASBench-301 (CIFAR-10), Raw lags behind initially but can catch up to one embedding variant in later stages. Among embeddings, no single method dominates across all cases; all three are effective and often comparable, with graph2vec occasionally exhibiting slightly better stability. Raw coding might suffice in simple or well-structured spaces, but in more complex or challenging search settings, embeddings typically accelerate convergence and improve final accuracy. Nevertheless, graph2vec provides several practical advantages:Simplicity and robustness: It is easy to implement, requires no training of separate neural network encoders, and works directly on the graph structure.Direct support for uncertainty quantification: By generating multiple embeddings with different initializations, we can estimate predictive uncertainty (e.g., via the standard deviation of predicted fitness), which is needed for our infill sampling strategy.Less hyperparameter tuning: Unlike GCN and transformer networks, graph2vec does not need careful adjustment of network size, learning rate, or training schedule, reducing engineering overhead. Simplicity and robustness: It is easy to implement, requires no training of separate neural network encoders, and works directly on the graph structure. Simplicity and robustness: It is easy to implement, requires no training of separate neural network encoders, and works directly on the graph structure. Direct support for uncertainty quantification: By generating multiple embeddings with different initializations, we can estimate predictive uncertainty (e.g., via the standard deviation of predicted fitness), which is needed for our infill sampling strategy. Direct support for uncertainty quantification: By generating multiple embeddings with different initializations, we can estimate predictive uncertainty (e.g., via the standard deviation of predicted fitness), which is needed for our infill sampling strategy. Less hyperparameter tuning: Unlike GCN and transformer networks, graph2vec does not need careful adjustment of network size, learning rate, or training schedule, reducing engineering overhead. Less hyperparameter tuning: Unlike GCN and transformer networks, graph2vec does not need careful adjustment of network size, learning rate, or training schedule, reducing engineering overhead. Given these aspects, and the fact that its empirical performance is consistently on par with or slightly better than alternatives in terms of stability, we adopt graph2vec as our default embedding method for our framework. However, it is worth emphasizing that in current standard NAS benchmarks, the effect of the embedding choice is less critical than expected, and all three are viable options. Evaluation of GAN module and infill criterion To rigorously assess the contribution of other key components of GNE-NAS, supplementary experiments were conducted on NASBench-201 with the CIFAR-10 dataset, focusing on two elements: (i) the GAN-based augmentation module, and (ii) the uncertainty term in the infill criterion. In the first case, the GAN module, responsible for generating synthetic architectures during the search process, was removed. As shown in Table 5, this modification resulted in a notable decrease in standard deviation of accuracy by 0.06 compared to the full configuration. This outcome indicates that the GAN module plays a crucial role in training the surrogate model. In the second case, the uncertainty term was removed from the infill criterion, and only the predicted mean performance was used as the basis for selecting candidate architectures. This change also led to a drop in validation accuracy (− 0.02%) and a reduction in the STD by about 0.07%, representing a substantial decrease in the framework’s ability to explore promising yet uncertain regions, along with a decline in the quality of uncertainty estimation. These results demonstrate that both the GAN-based augmentation module and the uncertainty-aware infill criterion make distinct and positive contributions to search effectiveness, and that their inclusion is essential for achieving the performance reported in the full GNE-NAS configuration.Table 5Performance comparison of two versions of GNE-NAS in NASBench-201 (cifar10).MethodSearch budgetValidationGNE-NAS w/o GAN10091.58 ± 0.11GNE-NAS w/o infill10091.56 ± 0.12GNE-NAS10091.58 ± 0.05 Table 5Performance comparison of two versions of GNE-NAS in NASBench-201 (cifar10).MethodSearch budgetValidationGNE-NAS w/o GAN10091.58 ± 0.11GNE-NAS w/o infill10091.56 ± 0.12GNE-NAS10091.58 ± 0.05 Performance comparison of two versions of GNE-NAS in NASBench-201 (cifar10). Conclusions This paper aims to approximate the true accuracy of trained neural networks using a neural predictor as a surrogate model. Utilize the graph2vec model to represent each neural network into 128-dimensional vectors. Additionally, for enhancing the search capability and effective updating of the surrogate model, the WGAN-GP network is employed to generate similar samples. Furthermore, online learning significantly contributes to keeping the surrogate model up to date. Moreover, using the particle swarm optimization algorithm in the optimal architecture search process enables the GNE-NAS algorithm to achieve competitive performance in the NasBench101, NasBench201 and NasBench301 search spaces. Future research is needed to apply the GNE-NAS method to other tasks like segmentation, object detection, digital image denoising, and Natural Language Processing (NLP). Moreover, a promising research direction involves developing complementary sampling techniques and adaptive environmental selection strategies to align with diverse search spaces.