Federated Learning-based Semantic Segmentation for Lane and Object Detection in Autonomous Driving
Gharbi Khamis Alshammari, Ahmad Abubakar, Nada M.O. Sid Ahmed, Naif Khalaf Alshammari · 2025
Read Federated Learning-based Semantic Segmentation for Lane and Object Detection in Autonomous Driving by Gharbi Khamis Alshammari, Ahmad Abubakar, Nada M.O. Sid Ahmed, Naif Khalaf Alshammari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18939.txt.
Introduction
Gharbi Khamis Alshammari and co-authors built in 2025 a privacy-preserving learning framework for self-driving cars. Cars need precise lane and object detection, but training one central model on pooled data raises privacy and scaling problems. Their approach trains models across many cars without moving the raw data.
The framework pairs a hybrid segmentation network with noise-adding privacy tools that guard model updates against inference attacks. It was tested on simulated driving data in two forms. Accuracy rose steadily over many rounds of shared training, and total loss fell by more than half. The privacy tools cost only a small amount of accuracy. The authors present this as a scalable path for fleets to learn together safely.
Catalog record
| CREATOR | Gharbi Khamis Alshammari, Ahmad Abubakar, Nada M.O. Sid Ahmed, Naif Khalaf Alshammari |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18939
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Federated Learning-based Semantic Segmentation for Lane and Object Detection in Autonomous Driving Gharbi Khamis Alshammari, Ahmad Abubakar, Nada M.O. Sid Ahmed, Naif Khalaf Alshammari Abstract Autonomous Vehicles (AVs) require precise lane and object detection to ensure safe navigation. However, centralized deep learning (DL) approaches for semantic segmentation raise privacy and scalability challenges, particularly when handling sensitive data. This research presents a new federated learning (FL) framework that integrates secure deep Convolutional Neural Networks (CNNs) and Differential Privacy (DP) to address these issues. The core contribution of this work involves: (1) developing a new hybrid UNet-ResNet34 architecture for centralized semantic segmentation to achieve high accuracy and Mean Intersection over Union (Mean IoU) that tackles privacy concerns due to centralized training (2) Implementing the privacy-preserving FL model, distributed across AVs to enhance performance through secure CNNs and DP mechanisms. In the proposed FL framework, the methodology systematically distinguishes itself from the existing approach through the following: (i) ensuring data decentralization through FL to uphold user privacy by eliminating the need for centralized data aggregation; (ii) integrating DP mechanisms to secure sensitive model updates against potential adversarial inference attacks; and (iii) evaluating the framework’s performance and generalizability using RGB and semantic segmentation (SEG) datasets derived from the CARLA simulator. Experimental results show significant improvements in accuracy, from 81.5% to 88.7% for the RGB dataset and from 79.3% to 86.9% for the SEG dataset over 20 to 70 Communication Rounds (CRs). Global loss was reduced by over 60%, and minor accuracy trade-offs from DP were observed. This study contributes by offering a scalable, privacy-preserving FL framework tailored for AVs, optimizing communication efficiency while balancing performance and data security. Please note: Abbreviations should be introduced at the first mention in the main text – no abbreviations lists. Suggested structure of main text (not enforced) is provided below. Introduction AVs rely on accurate lane and object detection to ensure safe navigation in complex driving environments [ 1 ] . Semantic segmentation, which assigns labels to every pixel in an image, is a crucial task for AV perception systems. However, real-world scenarios present challenges due to variations in road conditions, lighting, weather, and occlusions, which make achieving high segmentation accuracy difficult [ 2 , 3 , 4 , 5 ] . Current centralized DL approaches for semantic segmentation require large amounts of sensitive driving data to be stored on central servers, raising privacy concerns and potential regulatory violations [ 6 , 7 , 8 ] . While centralized models like UNet, ResNet, and SegNet have shown strong performance for lane and object detection, they present significant drawbacks. Privacy concerns arise when sensitive data is shared and stored centrally [ 9 ] . These methods also require substantial computational resources, limiting their scalability in real-world AV environments. Furthermore, existing models struggle with non-Independent and Identically Distributed (non-IID) data, where each AV captures data under different conditions, causing inconsistencies in model performance [ 10 ] . Addressing these challenges requires a decentralized solution capable of maintaining privacy, optimizing communication efficiency, and accommodating the computational limitations of AVs. Our approach, FL, offers a promising direction by enabling model training across multiple AVs without sharing raw data. However, implementing secure FL for semantic segmentation comes with its own challenges, including handling non-IID data, ensuring robust privacy through Differential Privacy (DP), and maintaining communication efficiency [ 11 ] . Several DL architectures have been proposed for semantic segmentation in AVs, such as UNet, ResNet34, and SegNet, which rely on centralized training [ 12 ] . Although these methods achieve high accuracy in controlled environments, their centralized nature introduces substantial privacy risks and poses significant challenges in terms of computational demand. The aggregation of large datasets on a central server raises security issues, particularly under strict privacy regulations like GDPR and CCPA [ 13 , 14 ] . Additionally, these architectures perform poorly with non-IID data, commonly observed in AV scenarios, leading to suboptimal convergence and degraded performance. In this study, we propose an enhanced FL framework that integrates secure deep CNNs and Differential Privacy (DP) techniques to address these challenges. The key claims of our proposed framework include: (i) Data decentralization via FL: FL allows AVs to collaboratively train models without sharing raw data, significantly reducing privacy risks. (ii) Differential Privacy integration: DP ensures that model updates are protected, further safeguarding sensitive information while maintaining robust performance. (iii) Hybrid architecture: We combine UNet’s segmentation capabilities with ResNet34’s feature extraction strengths, creating a hybrid model for accurate lane and object detection. (iv) Communication efficiency: Optimized communication protocols are implemented to reduce bandwidth usage and latency, essential for real-time AV environments. This research aims to improve semantic segmentation in AVs by addressing the aforementioned challenges. The main contributions of this research are as follows: 1. Hybrid UNet-ResNet34 for FL-Based Semantic Segmentation: The proposed hybrid UNet-ResNet34 model addresses the lack of adaptability in standard segmentation models (e.g., UNet, ResNet34) for non-IID federated learning scenarios by combining UNet’s fine-grained segmentation capabilities with ResNet34’s deep feature extraction, achieving superior performance in diverse lighting and environmental conditions on AV datasets compared to conventional architectures. 2. Privacy-Preserving Federated Learning (FL) Framework: Our FL framework eliminates data-sharing risks by training models directly on autonomous vehicles (AVs) instead of relying on centralized data aggregation, ensures decentralized learning while maintaining segmentation accuracy across heterogeneous AV datasets, and achieves comparable performance with significantly reduced privacy risks, as demonstrated by experimental results. 3. Differential Privacy (DP) for Secure Model Updates: To further enhance security, we integrate Differential Privacy (DP) mechanisms into Federated Learning (FL) to mitigate adversarial inference risks by adding controlled noise to model updates, ensuring privacy while maintaining high segmentation performance, with evaluations demonstrating that our DP-FL approach retains over 90% of the original accuracy alongside strong privacy guarantees. 4. Communication-Efficient FL for Scalable AV Networks: Federated Learning (FL) introduces communication overhead due to repeated model updates, but by optimizing communication protocols to reduce bandwidth consumption without sacrificing accuracy, our results demonstrate that increasing communication rounds (CRs) from 20 to 70 improves accuracy from 81.5% to 88.7% on the RGB dataset and from 79.3% to 86.9% on the SEG dataset, achieving these enhancements without excessive communication costs and thereby making FL feasible for real-world autonomous vehicle (AV) applications. 5. Robust Evaluation Across AV Datasets: Our model, benchmarked on RGB and SEG datasets from the CARLA simulator, a widely used AV perception dataset, demonstrates robustness through performance metrics (Mean IoU, sensitivity, specificity, F1-score) and outperforms baseline UNet, ResNet34, and SegNet models in both centralized and federated learning settings. Our proposed approach is distinct with the existing approaches through several points, including: (a) Eliminates centralized data storage while maintaining segmentation accuracy. (b) Ensures strong privacy protection using FL and DP techniques. (c) Optimizes communication efficiency, making FL viable for AV networks. (d) Scales effectively across non-IID AV datasets, outperforming standard FL and DL models. The remainder of this paper is organized as follows: Section 2 (Literature Review) discusses existing semantic segmentation methods, federated learning frameworks, and privacy-preserving techniques while highlighting their limitations; Section 3 (Materials and Methods) presents details on dataset selection, preprocessing, model architecture, the FL framework, and differential privacy integration; Section 4 (Experimental Setup) describes the experimental setup, implementation of the proposed