Concurrent Vertical and Horizontal Federated Learning with Fuzzy Cognitive Maps
Jose L. Salmeron, Irina Arévalo · 2024
Read Concurrent Vertical and Horizontal Federated Learning with Fuzzy Cognitive Maps by Jose L. Salmeron, Irina Arévalo (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12844.txt.
Introduction
Jose Salmeron and Irina Arévalo combine vertical and horizontal federated learning using fuzzy cognitive maps. Privacy rules block pooling data in health and finance. Federated training learns jointly without moving raw records.
Trouble starts when partners hold different feature sets. Some share rows, others share columns, many share neither. Fuzzy maps model the links between variables flexibly. Four weighting strategies for partner updates were tested. Constant, accuracy, and ranking-based weights all reached the goals. Privacy and secrecy standards held throughout the trials. The framework widens who can join a federation. It suits hospitals and banks with mismatched data and strict rules.
Catalog record
| CREATOR | Jose L. Salmeron, Irina Arévalo |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-12844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Concurrent Vertical and Horizontal Federated Learning with Fuzzy Cognitive Maps Jose L. Salmeron, Irina Arévalo Abstract Data privacy is a major concern in industries such as healthcare or finance. The requirement to safeguard privacy is essential to prevent data breaches and misuse, which can have severe consequences for individuals and organisations. Federated learning is a distributed machine learning approach where multiple participants collaboratively train a model without compromising the privacy of their data. However, a significant challenge arises from the differences in feature spaces among participants, known as non-IID data. This research introduces a novel federated learning framework employing fuzzy cognitive maps, designed to comprehensively address the challenges posed by diverse data distributions and non-identically distributed features in federated settings. The proposal is tested through several experiments using four distinct federation strategies: constant-based, accuracy-based, AUC-based, and precision-based weights. The results demonstrate the effectiveness of the approach in achieving the desired learning outcomes while maintaining privacy and confidentiality standards. Journal: Future Generation Computer Systems 1 Introduction Federated learning (FL) is an emerging distributed artificial intelligence framework that enables privacy-preserving machine learning by synthesizing local models instead of sharing actual data [ 1 ] . The general fundamental process can be outlined as follows [ 2 ] : the federation process is initiated by a single server or participant who provides an initial model for individual participants to train using their local data. These participants then share the model’s weights or gradients with the server (or other participants) for aggregation, typically using the average of the parameters. The federated model is iteratively transmitted back to the participants. This sequence continues until a predefined termination criteria are met. As a result, a federated model trained using the private data contributed by all participants is produced. This approach is more relevant when addressing sensitive data, particularly in domains like healthcare or finance. Moreover, the need for federated learning with non-independent and identically distributed (non-IID) data has arisen and grown strongly in several industries in recent years. Companies and institutions owning only small and fragmented data have been forming consortia with compensating data partners to collaboratively train machine learning models, but they may have different features space, or these features may have different distributions. In this research, the authors present a new federated learning framework with fuzzy cognitive maps (FCM) that is able to address the challenges posed by diverse data distributions and non-identically distributed features in federated settings. In the context of Fuzzy Cognitive Maps (FCMs), federated learning (FL) is employed to address several intrinsic challenges associated with these models. FL offers an effective approach to managing these challenges, enhancing the performance and applicability of FCMs. One necessary issue in the FCMs is the decentralised nature of data sources and the need to preserve data privacy and security while enabling collaborative model development. FCM models frequently rely on data distributed across multiple locations or organisations. Each participant may have unique datasets with varying characteristics, such as different samples and feature spaces. Moreover, datasets may contain sensitive information that participants are hesitant to share due to privacy and security concerns. Building accurate and generalised FCM models requires access to diverse datasets representing different perspectives and contexts. Therefore, the main contributions of this paper are three-fold: 1. An approach to distributed learning addressing both horizontal and vertical datasets; hence, it is called square federated learning. The introduction of a particle swarm optimisation-based fuzzy cognitive map learning method is presented, specifically designed for datasets containing both shared and distinct, or non-IID, features. To the best of our knowledge, such an approach, where participants have different samples and feature spaces, has not been addressed before. Therefore, this paper focuses on FCMs and the issues resolved with this approach rather than on the accuracy of the training. 2. A privacy-preserving machine learning strategy for FCMs is proposed. In FCMs, concepts, variables, or features are modelled as nodes, relationships between them as edges, and the weights represent the influence of these relationships. A training framework for collaborative FCM training prioritising data confidentiality was developed. This approach enables multiple participants to jointly train an FCM model using their individual data while ensuring strict compliance with data privacy regulations. 3. The proposal is tested with four federation strategies: constant-based, accuracy-based, AUC-based, and precision-based weights. This proposal is validated using well-known open datasets. The experimental outcomes confirm the effectiveness of the proposal, demonstrating its successful implementation and its potential to facilitate both horizontal and vertical FL using FCMs. The rest of this paper is organised as follows: Section 2 delves into the examination of related works. Section 3 presents the necessary information about fuzzy cognitive maps and federated learning. The methodological proposal is presented in Section 4 . Detailed insights into the experimental methodology and outcomes are outlined in Section 5 . Ultimately, the conclusions can be found in Section 6 . 2 Related work According to Ma et al. [ 3 ] , the existing methods to solve the non-IID problem of FL can be enclosed in four areas: data-based, model-based, algorithm-based and framework-based solutions. In the data-based space, Zhao [ 4 ] proposes a previous model trained with shared data in a central server in order to share a subset of shared datasets to each client, while Itahara [ 5 ] shares unlabeled data to protect privacy. Collins et al. [ 6 ] , train personalised low-dimensional classifiers for each client to learn shared data representations. Other authors propose data augmentation to weaken the effect of statistical heterogeneity [ 7 , 8 ] . To alleviate the problems data heterogeneity produces on the model, several authors have introduced changes in the model architecture [ 9 ] , the aggregation update method [ 10 ] , or the optimisation [ 11 , 12 ] . With respect to algorithm-based solutions, Jiang [ 13 ] proposes a model-agnostic meta-learning setting using federated average as the meta-learning algorithm for training and fine-tuning to personalize the model, while Li [ 14 ] introduces a meta-learning method for spatio-temporal problems. Smith et al. [ 15 ] , propose a new system-aware optimisation method called MOCHA that learns different tasks and captures their internal relationships in a multi-task setting. Finally, some authors have used that the impact of data heterogeneity on the model can be reduced by client similarity clustering. Ghosh [ 16 ] proposes a client-side clustering method that finds an independent local optimal solution for each client, sends it to the server and the server clusters the clients according to that solution. Briggs [ 17 ] introduces a clustering step after several iterations to hierarchically cluster client based on the similarity between the client’s local update and the global federated model. To the best of our knowledge, this work is the first one to exploit the characteristics of fuzzy cognitive maps for solving the heterogeneity problem in federated learning. 3 Fundamentals The integration of fuzzy cognitive maps and federated learning constitutes a synergistic approach that has the potential to significantly advance collaborative machine learning paradigms. FCMs are renowned for their capability to model complex relationships and dynamics within systems, offering a versatile toolset for capturing complex interrelations among variables. Meanwhile, federated learning emerges as an