Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems

Abdulmomen Ghalkha, Zhuojun Tian, Chaouki Ben Issaid, Mehdi Bennis · 2025

Read Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems by Abdulmomen Ghalkha, Zhuojun Tian, Chaouki Ben Issaid, Mehdi Bennis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22374.txt.

Introduction

Abdulmomen Ghalkha and co-authors propose, in a 2025 paper, a learning scheme that lets wireless edge devices train together across differing data types without a central server. Future networks swarm with sensors seeing images, sound, and signals at once. Standard federated learning assumes one data type and identical models everywhere.

Their framework borrows sheaf theory, a mathematics of gluing local data into global wholes. Each device keeps encoders suited to its own senses. Local views are stitched into shared understanding through the sheaf structure. Devices with modest hardware are not forced into a common mold. Richer joint awareness could sharpen the decisions next-generation networks make at their edges.

Catalog record

CREATORAbdulmomen Ghalkha, Zhuojun Tian, Chaouki Ben Issaid, Mehdi Bennis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-22374

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems Abdulmomen Ghalkha, Zhuojun Tian, Chaouki Ben Issaid, Mehdi Bennis Abstract In large-scale communication systems, increasingly complex scenarios require more intelligent collaboration among edge devices collecting various multimodal sensory data to achieve a more comprehensive understanding of the environment and improve decision-making accuracy. However, conventional federated learning (FL) algorithms typically consider unimodal datasets, require identical model architectures, and fail to leverage the rich information embedded in multimodal data, limiting their applicability to real-world scenarios with diverse modalities and varying client capabilities. To address this issue, we propose Sheaf-DMFL, a novel decentralized multimodal learning framework leveraging sheaf theory to enhance collaboration among devices with diverse modalities. Specifically, each client has a set of local feature encoders for its different modalities, whose outputs are concatenated before passing through a task-specific layer. While encoders for the same modality are trained collaboratively across clients, we capture the intrinsic correlations among clients’ task-specific layers using a sheaf-based structure. To further enhance learning capability, we propose an enhanced algorithm named Sheaf-DMFL-Att, which tailors the attention mechanism within each client to capture correlations among different modalities. A rigorous convergence analysis of Sheaf-DMFL-Att is provided, establishing its theoretical guarantees. Extensive simulations are conducted on real-world link blockage prediction and mmWave beamforming scenarios, demonstrate the superiority of the proposed algorithms in such heterogeneous wireless communication systems. I Introduction Machine learning (ML) has emerged as a key enabler for next-generation wireless communication systems [ 1 ] , tackling core challenges such as blockage prediction [ 2 , 3 ] , millimeter wave (mmWave) beamforming [ 4 , 5 , 6 ] , and channel estimation [ 7 ] . While these methods offer promising performance, their success heavily depends on access to large, high-quality datasets. In large-scale wireless networks, however, data is often distributed across numerous devices, each with limited and non-identically distributed samples. Federated learning (FL) has been introduced as a promising alternative, enabling multiple clients to collaboratively train ML models without sharing their raw data [ 8 , 9 ] . By preserving data privacy and reducing the need for centralized data aggregation, FL offers an attractive solution for collaborative intelligence in wireless communication problems [ 10 , 11 , 12 ] . Fig. 1 : Illustration of multimodal use cases in large-scale wireless communication systems: (a) Multimodal-aided blockage prediction, (b) Multimodal beam prediction. Despite notable advancements in FL, the majority of existing research is constrained to unimodal learning settings, where all clients operate on the same data modality, while employing identical model architectures [ 13 ] . However, this assumption fails to reflect the complexity and heterogeneity of real-world wireless communication systems, where clients observe data through diverse modalities, including visual inputs, acoustic data, or RF signals. Figure 1 provides a concrete motivating example illustrating two typical multimodal wireless communication scenarios. In scenario (a), a multimodal-aided blockage prediction setting is shown, where various sensing inputs are used to predict potential link disruptions/blockages. Scenario (b) depicts a beam prediction task, where the device utilizes multimodal data such as LiDAR and images to predict the optimal beam direction toward a moving drone. The benefits of multimodal learning in communication systems have also been demonstrated in prior works [ 14 , 15 , 7 ] . Specifically, results in [ 7 ] validate that combining received signal strength, user location, and uplink channel state information significantly improves downlink channel prediction accuracy in massive MIMO systems. These findings underscore the need for distributed learning frameworks that can effectively harness multimodal information. Motivated by this, we aim to design a distributed multimodal learning framework that leverages diverse modalities in real-world communication settings. Several recent works have explored the challenges and opportunities of multimodal federated learning, particularly in the presence of modal heterogeneity among clients. For example, [ 16 ] introduced contrastive objectives to align representations across and within modalities using a public dataset, while [ 17 ] proposed Harmony, which disentangles the training into a two-stage process consisting of modality-wise federated learning followed by federated fusion. The authors in [ 18 ] proposed simultaneously optimizing both the blending of modalities and the aggregation weights of local models by adaptively measuring their overfitting and generalization behaviors. Furthermore, [ 19 ] introduced a dynamic multi-view graph to model and exploit relationships among clients for improved multimodal learning, specifically by adaptively capturing correlations among multimodal models to address modality discrepancy. Although these methods validate the benefits of incorporating multiple modalities, they typically rely on a central parameter server (PS) to coordinate training and often assume access to public datasets for cross-modal alignment. This centralized approach introduces a single point of failure, limits scalability, and may be infeasible in distributed environments like Internet of Things (IoT) networks, where devices can only communicate via peer-to-peer links. To address these limitations, [ 16 ] proposed DMML-KD, a fully decentralized multimodal FL framework, which uses knowledge distillation to facilitate knowledge sharing across heterogeneous modalities. While this approach mitigates some of the coordination and communication challenges, it suffers from instability in encoder alignment, as clients sharing the same modality may still learn divergent encoders due to the local knowledge distillation process and varying modality contexts. Consequently, there remains a pressing need for a fully decentralized multimodal learning framework that can (i) accommodate diverse client architectures and data modalities, and (ii) facilitate efficient information sharing and collaboration without reliance on a PS. Recent work by Ben Issaid et al. [ 20 ] has explored the application of sheaf theory in decentralized multi-task learning, demonstrating its effectiveness in modeling feature and sample heterogeneity across clients. Sheaf structures [ 21 ] , originating from algebraic topology, offer a mathematically principled framework for rigorously integrating local data or models across a network. However, this work does not consider the unique challenges posed by multimodal FL, where clients may have access to different combinations of data modalities. In this work, we extend the sheaf-theoretic approach to the decentralized multimodal federated learning (DMFL) and demonstrate its applicability across diverse 6G scenarios. Built upon the decentralized multi-task learning paradigm introduced in [ 20 ] , we propose Sheaf-DMFL, treating each unique modality combination as a separate task, and the sheaf structure is leveraged to enable nuanced collaboration across clients with heterogeneous sensors by explicitly modeling and learning consistency relationships between these differing tasks. Furthermore, by incorporating a local attention mechanism in the embeddings’ fusion, Sheaf-DMFL-Att is proposed as an enhanced variant, which improves multimodal representation learning and shows better predictive performance in complex scenarios. Our contributions can be summarized as follows: • We propose a partially-shared model architecture where clients collaboratively train common encoders for shared modalities. The modality-specific features are then fused and passed through a client-specific head layer, framing the problem as federated multi-task learning (FMTL) where inter-task relationships are learned using the sheaf theoretic principles. • We propose Sheaf-DMFL, incorporating a sheaf Laplacian to capture and learn the inter-task relationships across clients with varying modality availability, which is jointly optimized during training, facilitating adaptive and efficient collaboration among clients with diverse modalities in a principled and scalable manner. • Furthermore, we introduce the local attention mechanism for multimodal fusion, named Sheaf-DMFL-Att, which demonstrates improved performance in

Read the complete text