Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art

Sebastian Peitz and Sedjro Salomon Hotegni · 2024

Read Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art by Sebastian Peitz and Sedjro Salomon Hotegni (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01566.txt.

Introduction

Sebastian Peitz and Sedjro Salomon Hotegni survey multi-objective deep learning in a 2024 paper. Machine learning often must serve more than one goal at once. Extra goals can include sparse models or tuning choices. Deep networks make that task harder, with their many parameters, strong curves, and random training.

The survey sorts methods by training style and by what the decision maker needs. It covers supervised, unsupervised, and reinforcement learning, plus generative modeling. Recent gains and strong uses are listed for each group. The authors aim to give readers a clear map of a fast moving field and its open problems.

Catalog record

CREATORSebastian Peitz and Sedjro Salomon Hotegni
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01566

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art Sebastian Peitz and Sedjro Salomon Hotegni Abstract Simultaneously considering multiple objectives in machine learning has been a popular approach for several decades, with various benefits for multi-task learning, the consideration of secondary goals such as sparsity, or multicriteria hyperparameter tuning. However—as multi-objective optimization is significantly more costly than single-objective optimization—the recent focus on deep learning architectures poses considerable additional challenges due to the very large number of parameters, strong nonlinearities and stochasticity. This survey covers recent advancements in the area of multi-objective deep learning. We introduce a taxonomy of existing methods—based on the type of training algorithm as well as the decision maker’s needs—before listing recent advancements, and also successful applications. All three main learning paradigms supervised learning, unsupervised learning and reinforcement learning are covered, and we also address the recently very popular area of generative modeling. I Introduction Conflicting design or decision criteria are everywhere, and the challenge of identifying suitable decisions has been around for a very long time. To address this problem, we are looking for the set of optimal compromises—also called the Pareto set after the Italian economist Vilfredo Pareto. In simple terms, a decision is Pareto optimal if we are unable to further improve all objectives at the same time, but instead have to accept a trade-off if we want to further improve one of the criteria. In mathematical terms, this can be cast as a multi-objective optimization problem , see [ 120 , 43 ] for detailed introductions. Problems of this type have been studied in a large number of fields, examples being political decision-making [ 175 ] , medical therapy planning [ 94 ] , the design of numerical algorithms [ 185 ] , or the control of complex dynamical systems [ 136 ] . Very naturally, multiple criteria also arise in the area of machine learning, for instance if we want to solve multiple tasks with a single model, which results in several performance measures. Alternatively, we might be interested in secondary objectives such as sparsity/efficiency, robustness or interpretability. Specifically in the area of reinforcement learning, researchers have made strong arguments to intensify multi-objective research [ 146 , 184 ] . However, since both multi-objective optimization and deep learning can be computationally expensive, their combination gives rise to particular challenges in terms of algorithmic efficiency and decision-making. The goal of this survey paper is to provide a detailed overview of the state of the art in multi-objective deep learning. A large number of articles has appeared in recent years, and progress has been rapid. To keep an overview and ease the entry for researchers interested in—but until now less familiar with—multi-objective learning, we provide a taxonomy of the various approaches (Section III ), before surveying the current state of the art in Section IV . We consider the three main learning paradigms supervised learning ( IV-A ), unsupervised learning ( IV-B ) and reinforcement learning ( V —since this is a special case, we will discuss it separately), and we also provide an overview of generative modeling ( IV-C ), neural architecture search ( IV-D ) and successful applications of multi-objective deep learning ( IV-E ). Finally, we briefly comment on the vice-versa combination in Section VI , i.e., the usage of deep learning to enhance the solution of multi-objective optimization problems. II Preliminaries Before introducing our taxonomy and survey to multi-objective deep learning we here give brief introductions to deep learning ( II-A ) and multi-objective optimization ( II-B ), respectively and highlight the specifics of multi-objective machine learning in Section II-C . II-A Deep learning This section’s main purpose is to introduce notation and to highlight the similarity between various areas of machine learning when it comes to training and optimization. We briefly cover supervised ( II-A1 ) and unsupervised learning ( II-A2 ) as well as generative modeling ( II-A3 ). For much more detailed introductions, the reader is referred to one of the excellent text books [ 19 , 69 , 62 , 20 ] . II-A 1 Supervised learning The overarching theme in supervised learning is function approximation from examples. In other words, we want to approximate an unknown input-to-output mapping y=f(x) , f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m} , by a parametrized function f_{\theta} . Here, \theta\in\mathbb{R}^{q} represents the (usually high-dimensional) vector of trainable parameters. There exist many options how to construct f_{\theta} , such as polynomials, Fourier series, or Gaussian processes. The specialty in deep learning [ 96 ] is that f_{\theta} represents a deep neural network with \ell layers, in which affine transformations (W,b) alternate with nonlinear activation functions \sigma : \displaystyle z^{(0)} \displaystyle=x, \displaystyle z^{(j)} \displaystyle=\sigma^{(j)}\left(W^{(j)}z^{(j-1)}+b^{(j)}\right),\quad j=1,\ldots,\ell, \displaystyle y \displaystyle=z^{(\ell)}. All trainable weights are collected in \theta=\left\{\left(W^{(j)},b^{(j)}\right)\right\}_{j=1}^{\ell} . Remark 1 . Besides the just-mentioned classical feed-forward architecture, there are numerous other deep learning models used for function approximation such as convolutional networks , residual networks , or recurrent networks with feedback loops [ 62 , 20 ] . However, the goal of this article is not to cover the details of deep learning models, but to shed light on the training with multiple criteria, which is why we are not going into more detail here. To fit f_{\theta} to f , we use a dataset \mathcal{D}=\{(x_{i},y_{i})\}_{i=1}^{N} of N samples and minimize the empirical loss L : \displaystyle\theta^{*}=\arg\min_{\theta\in\mathbb{R}^{q}}L(\theta). (1) A common choice for L is the mean squared error, i.e., L(\theta)=\frac{1}{N}\sum_{i=1}^{N}\left\|y_{i}-f_{\theta}(x_{i})\right\|_{2}^{2}, but there are numerous alternatives as well as additional terms (e.g., for regularization or sparsity [ 97 ] ). Regardless of the specific choice, Problem ( 1 ) is a high-dimensional, nonlinear optimization problem. The typical approach to find \theta^{*} (or at least a \theta that yields a satisfactory performance) is gradient-based optimization using backpropagation, often in combination with stochasticity and momentum, as in the popular Adam algorithm [ 90 ] or adaptations thereof (e.g., [ 25 ] ). Consequently, the gradient \nabla L(\theta) plays a central role in training. II-A 2 Unsupervised and self-supervised learning Unlike supervised learning, which depends on labeled data to model input-output relationships, unsupervised and self-supervised learning focus on extracting meaningful representations or patterns from unlabeled data [ 156 , 164 ] . While unsupervised learning relies on directly identifying latent structures in the data [ 29 , 40 , 194 ] , self-supervised learning uses surrogate tasks, derived from the data itself, to create labels and train models that can generalize to downstream tasks [ 64 , 87 ] . The objectives in unsupervised learning often involve clustering [ 27 ] , dimensionality reduction [ 154 ] , or density estimation [ 193 ] , formulated as optimization problems. Clustering focuses on partitioning a dataset \mathcal{D}=\{(x_{i})\}_{i=1}^{N} into N_{C} clusters. A popular objective is to minimize intra-cluster variance while maximizing inter-cluster separation: \min_{C_{1},\dots,C_{K}}\sum_{k=1}^{N_{C}}\sum_{x_{i}\in C_{k}}\|x_{i}-\mu_{k}\|^{2},\quad\mu_{k}=\frac{1}{|C_{k}|}\sum_{x_{i}\in C_{k}}x_{i}, where C_{k} is the set of data points assigned to the k^{\text{th}} cluster, and \mu_{k} is its centroid. Techniques like K-means and Gaussian Mixture Models (GMMs) [ 143 ] address this optimization. Modern neural clustering approaches embed data into latent spaces, enhancing the flexibility and scalability of clustering methods [ 202 , 101 ] . For dimensionality reduction, the goal is to find a lower-dimensional representation

Read the complete text