Evolutionary Machine Learning meets Self-Supervised Learning: a comprehensive survey

Adriano Vinhas, João Correia, Penousal Machado · 2025

Read Evolutionary Machine Learning meets Self-Supervised Learning: a comprehensive survey by Adriano Vinhas, João Correia, Penousal Machado (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07213.txt.

Introduction

Adriano Vinhas, João Correia, and Penousal Machado present a 2025 survey at the meeting point of two machine learning fields. Evolutionary methods automate the design of learning algorithms by breeding and selecting them. Self-supervised learning, meanwhile, extracts useful features from unlabeled data, which matters when labels are scarce.

Papers combining the two have multiplied, yet no map of the territory existed. The authors chart the combined work, propose a taxonomy, and name the emerging subfield Evolutionary Self-Supervised Learning. Evolution can shape self-supervised pipelines, while self-supervision can guide evolution itself or cut labeling needs. The survey closes with open challenges and research directions for a young but fast-growing area.

Catalog record

CREATORAdriano Vinhas, João Correia, Penousal Machado
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-07213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evolutionary Machine Learning meets Self-Supervised Learning: a comprehensive survey Adriano Vinhas, João Correia, Penousal Machado Abstract Research that combines Evolutionary Machine Learning and Self-Supervised Learning has been steadily increasing in recent years. Evolutionary Machine Learning has been shown to help automate the design of machine learning algorithms and to lead to more reliable solutions. Self-Supervised Learning, on the other hand, has produced good results in learning useful features when labelled data is limited. This suggests that the combination of these two areas can help both in shaping evolutionary processes and in automating the design of deep neural networks, while also reducing the need for labelled data. However, no detailed surveys exist that explain how Evolutionary Machine Learning and Self-Supervised Learning can be used together. To help with this, we provide an overview of studies that bring these areas together. Driven by the growing interest in this topic and the range of existing works, we consolidate current knowledge on approaches that combine these two fields, propose a definition for this area of research, which we call Evolutionary Self-Supervised Learning, and introduce a taxonomy for it. Finally, we point out some of the main challenges and suggest directions for future research to help Evolutionary Self-Supervised Learning grow and mature as a field. Journal: Swarm and Evolutionary Computation 1 Introduction Evolutionary Machine Learning (EML) [ 1 , 2 , 3 , 4 ] is a field that has attracted the attention of researchers due to its ability to produce competitive, and often unexpected, Machine Learning (ML) models that solve a given problem. This ability comes from the fact that EML algorithms are stochastic and, therefore, able to promote the emergence of robust solutions under noisy conditions. Additionally, since they don’t rely on human-made solutions, they may find unexpected ones. As the field evolves, recent developments have demonstrated competitive or superior performance compared to traditional methods, particularly in terms of generalisation, adaptability, interpretability and efficiency. In parallel, the use of the Self-Supervised Learning (SSL) paradigm has become widespread due to its ability to use unlabelled data to learn representations, which is particularly useful given that the process of data labelling is a task that can be tedious, time-consuming, and prone to errors. Within the text domain, BERT [ 5 ] set new records in several benchmarks by pretraining the model without labels. This success promoted the research in designing pretext tasks with similar impact in computer vision [ 6 ] , audio [ 7 ] , video [ 8 ] , or in multimodal scenarios [ 9 ] . The growing interest in these fields has sparked a line of research that merges EML and SSL . This combination can bring benefits by increasing the robustness of the representations learned by SSL algorithms or improving the evolution process behind the learned solutions, especially in situations where labelled data is limited. The surveys about EML that were identified [ 2 , 3 , 4 ] do not handle the topic of leveraging unlabelled data for representation learning despite listing a few studies that fit the subject. Similarly, surveys on SSL [ 10 , 11 ] fall short in terms of detailing the efforts to automate the search or design of pretext tasks. Interestingly, conda activate neat covers the intersection of Supervised Learning with EML [ 12 ] . However, to the best of our knowledge, no survey focuses on works that intersect EML and SSL with such a broad view. Nevertheless, we have identified a survey that focuses specifically on the use of Evolutionary Computation (EC) in the context of Generative Adversarial Networks (GANs) [ 13 ] . Since a dedicated survey already covers this topic, we exclude it from our scope and direct readers to it for further details. In the scope of this paper, we focus on studies that combine EML and SSL , defining a field which encompasses all the works that fit within this intersection, naming it Evolutionary Self-Supervised Learning (E-SSL) . In total, we identified 72 papers that fall within the scope of this work. The number of these publications by year is shown in Figure 1 , and it supports the idea that interest in this area is growing. To find these papers, we used search terms related to EC together with terms related to SSL , using the AND operator. Some of the EC terms include: evolutionary, evolutionary neural architecture search, genetic programming, evolutionary strategies, and cma-es. For the SSL keywords, we used terms like self-supervised, pretext task, autoencoder, contrastive, siamese networks, and unsupervised pretraining. We ran our searches on Google Scholar, IEEExplore, ACM Digital Library, and SpringerLink. Figure 1: Number of surveyed publications related with E-SSL breakdown by year The main contributions of this survey are: (i) the formalisation of Evolutionary Self-Supervised Learning (E-SSL) as a research area, together with a taxonomy derived from recent studies; and (ii) a discussion of open challenges and promising directions for future work within this taxonomy. The remainder of this paper is organised as follows: Section 2 gives background on the SSL field, while Section 3 provides an overview of the EML field; Section 4 describes the new research area and the proposed taxonomy; Section 5 covers open challenges and future research directions; Finally, Section 6 wraps up the paper by summarising the main contributions. 2 Self-Supervised Learning (SSL) SSL has been gaining popularity in recent years due to its ability to learn from large amounts of unlabelled data, which is often easier to obtain than labelled data. It is a learning paradigm that uses the input data itself to create pseudo-labels that will guide the learning process. Pseudo-labels can be defined based on a property of the input, or generated from the relationship between the original input and a modified version of it. This modification can take the form of parts of the original input, a corrupted version (created by masking, adding noise, or applying some other distortion), or even a different modality of the data. The goal of SSL is to learn representations from signals derived from the inputs [ 14 ] . The SSL paradigm is particularly useful when the number of labelled inputs is limited, as it can be divided into two distinct stages. In the first stage, a model is trained without using any labels and without explicitly considering the final goal. Instead, pseudo-labels are used to supervise training, which forces the model to learn how to extract features for a task that is related to the actual goal but not the same. This initial step is known as the pretext task. The second stage, called the downstream task, uses the model trained in the pretext task to help solve the actual target task. This stage follows a Supervised Learning setting, where features extracted from labelled inputs are used to train the model that solves the target task. The representations learned during the pretext task can either be reused directly or fine-tuned during this second step. An overview of this two-step process in SSL is shown in Figure 2 . Figure 2: Overview of the SSL process for an image classification problem. The success of SSL depends on the quality of the representations learned during the

Read the complete text