Election of Collaborators via Reinforcement Learning for Federated Brain Tumor Segmentation

Muhammad Irfan Khan, Elina Kontio, Suleiman A. Khan, Mojtaba Jafaritadi · 2024

Read Election of Collaborators via Reinforcement Learning for Federated Brain Tumor Segmentation by Muhammad Irfan Khan, Elina Kontio, Suleiman A. Khan, Mojtaba Jafaritadi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20253.txt.

Description

Abstract Federated learning (FL) enables collaborative model training across decentralized datasets while preserving data privacy. However, optimally selecting participating collaborators in dynamic FL environments remains challenging. We present RL-HSimAgg, a novel reinforcement learning (RL) and similarity-weighted aggregation (simAgg) algorithm using harmonic mean to manage outlier data points.

Catalog record

CREATORMuhammad Irfan Khan, Elina Kontio, Suleiman A. Khan, Mojtaba Jafaritadi
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-20253

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Election of Collaborators via Reinforcement Learning for Federated Brain Tumor Segmentation Muhammad Irfan Khan, Elina Kontio, Suleiman A. Khan, Mojtaba Jafaritadi Abstract Federated learning (FL) enables collaborative model training across decentralized datasets while preserving data privacy. However, optimally selecting participating collaborators in dynamic FL environments remains challenging. We present RL-HSimAgg, a novel reinforcement learning (RL) and similarity-weighted aggregation (simAgg) algorithm using harmonic mean to manage outlier data points. This paper proposes applying multi-armed bandit algorithms to improve collaborator selection and model generalization. By balancing exploration-exploitation trade-offs, these RL methods can promote resource-efficient training with diverse datasets. We demonstrate the effectiveness of Epsilon-greedy (EG) and upper confidence bound (UCB) algorithms for federated brain lesion segmentation. In simulation experiments on internal and external validation sets, RL-HSimAgg with UCB collaborator outperformed the EG method across all metrics, achieving higher Dice scores for Enhancing Tumor (0.7334 vs 0.6797), Tumor Core (0.7432 vs 0.6821), and Whole Tumor (0.8252 vs 0.7931) segmentation. Therefore, for the Federated Tumor Segmentation Challenge (FeTS 2024), we consider UCB as our primary client selection approach in federated Glioblastoma lesion segmentation of multi-modal MRIs. In conclusion, our research demonstrates that RL-based collaborator management, e.g. using UCB, can potentially improve model robustness and flexibility in distributed learning environments, particularly in domains like brain tumor segmentation. Federated learning (FL) represents a decentralized and democratic framework for collaborative machine learning (ML). Under this framework, various independent entities with unique datasets and compute resources collaborate without sharing data. These entities, referred to as collaborators , function as autonomous units, safeguarding the integrity of their data while working towards a common objective [ 15 , 13 ] . Just as groups come together for collective decision-making, collaborators in FL engage in a cooperative effort to train a shared ML model. However, unlike centralized ML frameworks, FL is characterized by a distributed nature, akin to a network of independent entities. Collaborators maintain local control over their private or sensitive data and training processes, reflecting their independence within the federated framework. An FL system encourages collaboration and representation, similar to systems that prioritize participation. Collaborators are chosen for training rounds based on their past contributions to model accuracy, efficiency, and overall improvement. This feedback can influence their selection for future rounds, ensuring that the most effective collaborators are prioritized. Furthermore, the aggregation of model updates resembles a collective consensus-building process, as collaborators contribute their insights to shape the collective model. This flexible approach accommodates differences in data, similar to variations in preferences across different entities. A cornerstone of FL’s systems is balancing the needs of each collaborator with the objective of improving the model as a whole. FL facilitates collaboration and equitable participation, encourages the exchange of knowledge, and advances ML while protecting the privacy of data [ 10 , 14 , 13 ] . Efficiently selecting collaborators for federation rounds presents a complex challenge [ 19 , 18 , 8 ] . Traditionally, a batch-wise approach has been the standard protocol, where available collaborators are preselected for each round of FL [ 7 ] . However, this approach often encounters limitations in adapting to dynamic scenarios where new collaborators may join with evolving data distributions and hardware configurations [ 1 ] . This limitation has motivated the exploration of alternative methods, such as multi-armed bandit algorithms, to enhance the collaborator selection process [ 12 , 2 ] . Optimal model training and improved generalization on unseen data can be achieved through performance-based selection of the best possible collaborators. To realize this objective, candidate profiles of collaborators are maintained in each FL round, and collaborators are selected using reinforcement learning (RL), employing a Markov decision process [ 20 ] . This introduces healthy competition among collaborators, ensuring effective collaboration. In healthcare and other dynamic fields, the federated learning landscape is constantly evolving. New potential collaborators, each with unique data distributions and hardware setups, frequently join the network. This diversity is often driven by the influx of new patient data at the start of federation rounds, making adaptability a key feature of the system [ 21 ] . In this context, the application of multi-armed bandit algorithms appears well-suited and adaptable to address this challenge. Here, each collaborator can be likened to a "bandit," and the process of selecting the most promising collaborator can be framed as an optimization problem. The criteria for making this selection can be fine-tuned by striking a balance between exploration and exploitation, effectively managed through a hyperparameter. This hyperparameter can guide the training of the master model at the aggregator, offering the flexibility needed to optimize the selection process. One notable advantage of this approach is its ability to mitigate or eliminate system-level bias in collaborator selection during federation rounds, which is often associated with the automatic grouping of collaborators in batch-wise techniques. This work highlights the application of RL for collaborator selection in federated brain tumor segmentation. We leverage Epsilon-greedy (EG) and Upper Confidence Bound (UCB) algorithms to optimize collaborator selection, aiming to improve the efficiency and effectiveness of the FL process. This approach represents a novel application of RL in the domain of brain lesion segmentation, potentially leading to significant advancements in this crucial clinical imaging field. 1 Methods 1.1 Data The current study utilized real-world multi-parametric magnetic resonance imaging (mpMRI) data from glioblastoma (GBM) cases made publicly available through the Federated Tumor Segmentation (FeTS) 2022 challenge. GBM is a clinically aggressive brain malignancy classified as a grade IV astrocytoma [ 9 ] . The dataset encompassed 1251 mpMRI scans from confirmed GBM patients, including native T1-weighted images, post-gadolinium contrast enhanced T1-weighted images, T2-weighted images, and T2 Fluid Attenuated Inversion Recovery (FLAIR) images. These cases were derived from the Brain Tumor Segmentation (BraTS) Continuous Challenge dataset, which aims to promote the development and benchmarking of state-of-the-art algorithms for consistent brain tumor delineation across institutions. Specifically, the GBM cohort examined in this study represented a subset of diffuse glioma cases from BraTS containing ground truth segmentations of tumor sub-regions (edema, enhancing core, and necrotic core). By utilizing real-world training data from FeTS and BraTS, we were able to evaluate federated learning for robust GBM segmentation under controlled conditions with known ground truth labels [ 3 , 4 , 5 , 6 , 16 ] . The mpMRI data utilized in this study was sourced from multiple academic institutions and preprocessed according to standard protocols including rigid registration, brain extraction, alignment, resolution resampling to uniform voxel sizes, and skull stripping for isolation of intracranial content. To develop and evaluate a federated learning approach for collaborative training of a segmentation model without sharing patient data, we implemented Intel’s Federated Learning OpenFL framework along with a U-Net convolutional neural network (CNN) architecture provided through the FeTS 2022 challenge. The federated learning methodology employed a randomly selected subset of participating sites to train local models and update the global model in each round - approximately 20% of total sites actively contributed to the global model training in a given round. This ensured that no one site had perpetual control over model updates, promoting decentralized collaboration. 1.2 Multi-Armed Bandit Problem In a stochastic multi-armed bandit problem, an agent must choose between K actions ("arms") in order to maximize expected

Read the complete text