Multi-objective methods in Federated Learning: A survey and taxonomy

Maria Hartmann, Grégoire Danoy, Pascal Bouvry · 2025

Read Multi-objective methods in Federated Learning: A survey and taxonomy by Maria Hartmann, Grégoire Danoy, Pascal Bouvry (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03108.txt.

Introduction

Maria Hartmann, Grégoire Danoy, and Pascal Bouvry survey how multi objective methods meet federated learning, in a 2025 paper. Federated learning trains models across many holders of data without pooling it. Real tasks in that setting must balance clashing goals such as fairness, accuracy, and resource use.

A multi objective lens fits that clash, but no one had mapped the union of the two fields. The authors offer the first taxonomy of the ways they combine, with clear labels for each kind of work. The scheme is built to absorb future studies as the area grows. Open problems and next steps close the survey, giving newcomers a firm place to start.

Catalog record

CREATORMaria Hartmann, Grégoire Danoy, Pascal Bouvry
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03108

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-objective methods in Federated Learning: A survey and taxonomy Maria Hartmann, Grégoire Danoy, Pascal Bouvry Abstract The Federated Learning paradigm facilitates effective distributed machine learning in settings where training data is decentralized across multiple clients. As the popularity of the strategy grows, increasingly complex real-world problems emerge, many of which require balancing conflicting demands such as fairness, utility, and resource consumption. Recent works have begun to recognise the use of a multi-objective perspective in answer to this challenge. However, this novel approach of combining federated methods with multi-objective optimisation has never been discussed in the broader context of both fields. In this work, we offer a first clear and systematic overview of the different ways the two fields can be integrated. We propose a first taxonomy on the use of multi-objective methods in connection with Federated Learning, providing a targeted survey of the state-of-the-art and proposing unambiguous labels to categorise contributions. Given the developing nature of this field, our taxonomy is designed to provide a solid basis for further research, capturing existing works while anticipating future additions. Finally, we outline open challenges and possible directions for further research. 1 Introduction The Federated Learning (FL) paradigm allows the training of machine learning models in the difficult setting where training data is distributed and compartmentalised. Instead of centralising available data, FL performs local training in distribution, with the resulting local models aggregated periodically across participants. Though originally designed to mitigate privacy concerns, the method has also shown great success in other use cases, including communication-restricted settings such as drone networks Brik et al. (2020) or computationally costly settings such as the tuning of large language models Che et al. (2023) . However, as Federated Learning is being adopted for increasingly diverse applications and real-world use cases, new challenges are emerging, many linked to the need to balance different conflicting requirements: (i) Heterogeneity between participants caused by data imbalances or differing hardware capabilities can lead to divergent local models that cannot easily be aggregated without loss of model utility Karimireddy et al. (2019) . Designing mitigation strategies for this raises the problem of fairness – the choice between sacrificing the performance of some individual clients or that of the global model. (ii) The cost of FL in terms of communication and computation resources scales with the size of the model and the number of update messages; yet reducing either may come at the cost of decreasing model utility Zhu et al. (2021) . (iii) Strategies for mitigating privacy leakage, the problem of exposing confidential information to potential attackers through client updates, may degrade other aspects of the federated system in turn. For example, adding noise to client updates may obscure sensitive information effectively, but reduce model performance as well Geng et al. (2024) . All these scenarios can be modelled as multi-objective problems, with each problem-specific performance metric represented as a separate objective. Under this multi-objective perspective, problems are solved with explicit consideration for several characteristics, potentially conflicting, and solutions can represent different optimal trade-offs between all objectives. As such, the approach can assist users in making informed decisions about complex FL problems by presenting explicit choices where a single-objective approach would yield none. Indeed, these general advantages of multi-objective methods have been recognised across disciplines, and the field of multi-objective optimisation (MOO) has been thriving for decades. This success opens another interesting avenue of research in connection with federated learning: deploying FL methods to facilitate multi-objective learning in distribution, where problems would otherwise be difficult to solve for participants that cannot share local training data. Recent works in the literature have begun to combine federated learning with MOO methods to address a wide range of challenges. However, the broader context of the intersection between MOO and FL has not yet been discussed. This work aims to provide a first such systematic overview, identifying general challenges and parallels, and formulating a novel taxonomy to classify existing work while highlighting open directions of research. Many FL strategies already use (linear) combinations of multiple functions as objectives, but do not consider the problem from a multi-objective angle. The first works to explicitly introduce multi-objective methods to Federated Learning aimed to improve federated aggregation and introduce fairness between clients Hu et al. (2022) , followed by approaches introducing other, system-wide aggregation parameters Mehrabi et al. (2022) . Another early adoption of MOO was in hyperparameter optimisation for FL Zhu and Jin (2020) . More recently, research has also begun into supporting the inverse scenario: developing strategies to federate the solving of multi-objective problems by distributed clients, e.g. Yang et al. (2023) Hartmann et al. (2023) . The contributions of this work can be summarised thus: • We propose a novel taxonomy of algorithms combining MOO methods and FL, offering a unified naming system for works at the intersection of two previously largely separate fields with separate naming conventions. • We present a thorough review of the state of the art, categorising and contrasting existing works. • We highlight open questions and offer perspectives on open avenues for future research. The rest of this work is organised as follows: Section 2 reviews important notions from the fields of FL and MOO. Section 3 introduces our taxonomy, discussing in detail each category and relevant works from the literature. Finally, we offer a conclusion and perspectives on future research in Section 4 . 2 Background In this section, we briefly introduce fundamental concepts from the fields of federated learning and multi-objective optimisation in preparation for the main body of the survey. 2.1 Federated Learning The Federated Learning McMahan et al. (2017) paradigm was originally designed to solve arbitrary (neural network-based) machine learning problems in a difficult distributed setting. This setting is characterised by (i) the available data originating in distribution, with no control over the composition of the resulting datasets, and (ii) a restriction on transmitting private client information, including raw training data, between participants. FL overcomes the constraint introduced by (ii) by training separate local models in distribution on each dataset holder, or client, and aggregating only the resulting models across clients – see Figure 1 . A more detailed general framework of the Federated Learning strategy is presented in Algorithm 1 , with colours highlighting the correspondence of code segments to different levels of the federated system (to be presented in detail in Section 2.3 ). First, the federated system is initialised with the identity of the server, a list of participating clients, and the definition of the underlying learning problem to be solved. Additional hyperparameters are passed depending on the specific algorithm, defining e.g. the architecture of the neural network to be trained, a client sampling rate, gradient thresholds, or any other parameter required by the algorithm. Then, the local learning process begins. During each federated training round, a set of clients is selected for participation. These clients each carry out local training and return the resulting models to the server. These local models are aggregated periodically by the server into a single global model incorporating the locally learned information. The global model is then passed back to the local clients to continue the next local training round. Figure 1: The FL paradigm. During each round,

Read the complete text