WeatherSeg: Weather-Robust Image Segmentation using Teacher-Student Dual Learning and Classifier-Updating Attention

Zhang Zhang, Yifeng Zeng, Houshi Jiang, Yinghui Pan* · 2026

Read WeatherSeg: Weather-Robust Image Segmentation using Teacher-Student Dual Learning and Classifier-Updating Attention by Zhang Zhang, Yifeng Zeng, Houshi Jiang, Yinghui Pan* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22824.txt.

Introduction

Zhang and three colleagues present WeatherSeg, a semi-supervised segmentation framework for cars that drive themselves in bad weather. It also cuts the cost of hand labels. A dual teacher-student model distills knowledge from weather-degraded images, while an attention mechanism retunes classifier weights to suit current conditions. Across clear, rainy, cloudy, and foggy scenes it beats baselines on accuracy and robustness. The result is a single approach to all-weather semantic segmentation for driving and related uses.

Catalog record

CREATORZhang Zhang, Yifeng Zeng, Houshi Jiang, Yinghui Pan*
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52897
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22824

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WeatherSeg: Weather-Robust Image Segmentation using Teacher-Student Dual Learning and Classifier-Updating Attention Zhang Zhang is with the School of Artificial Intelligence and Information Engineering, Zhejiang University of Science and Technology, Hangzhou, China (email: [email protected]). Yifeng Zeng is with the Department of Computer and Information Sciences, Northumbria University, Newcastle, United Kingdom (email: [email protected]). Houshi Jiang is with Beijing Quinovare Medical Technology Co., Ltd, Beijing, China (email: [email protected]). Yinghui Pan is the corresponding author. She is with the School of Artificial Intelligence, Shenzhen University, Shenzhen, China (email: [email protected]). Zhang Zhang, Yifeng Zeng, Houshi Jiang, Yinghui Pan* Abstract WeatherSeg , an advanced semi-supervised segmentation framework, addresses autonomous driving’s environmental perception challenges in adverse weather while reducing annotation costs. This framework integrates a Dual Teacher-Student Weight-Sharing Model (DTSWSM) that enables knowledge distillation from weather-affected images, and a Classifier Weight Updating Attention Mechanism (CWUAM) that dynamically adjusts classifier weights based on environmental attributes. Comprehensive evaluations demonstrate that WeatherSeg significantly outperforms baseline models in both accuracy and robustness across various weather conditions, including clear, rainy, cloudy, and foggy scenarios, establishing it as an effective solution for all-weather semantic segmentation in autonomous driving and related applications. I Introduction Semantic Segmentation [ 9 ] , [ 22 ] , a cornerstone of computer vision, enables the pixel-wise classification of images into distinct categories. Recent deep learning breakthroughs have dramatically improved performance in benchmark datasets. This capability critically supports autonomous driving by enabling the comprehension of the road scene, accurate object detection, and reliable obstacle avoidance. However, real-world driving environments present substantial challenges due to constantly varying lighting and weather conditions. This environmental complexity makes comprehensive data annotation exceptionally difficult, severely limiting the generalisation capability of current perception models. Extreme weather conditions—including heavy rain, dense fog, and glaring sunlight—directly impair autonomous systems’ perception reliability. These conditions not only obscure visual clarity but also hinder accurate environmental assessment and timely decision-making. Furthermore, obtaining large-scale, clearly annotated datasets covering diverse extreme weather scenarios requires prohibitive cost and time investment. Many existing approaches attempt to enhance training sample diversity through data augmentation techniques. These methods seek to optimise sensor cost-effectiveness while improving model robustness across environments. Despite these efforts, autonomous vehicles still face a critical shortage of adequate training data for extreme weather operations. This shortage has shifted research focus toward developing lightweight, easily integrable frameworks for existing segmentation networks. Semi-supervised semantic segmentation (SSL) [ 9 ] networks offer a promising solution by reducing dependence on expensively labeled data. These networks leverage semi-supervised learning to improve data processing efficiency and enhance visual interpretation under adverse conditions. SSL techniques effectively balance abundant unlabelled data utilisation with limited labeled sample precision. Nevertheless, existing SSL methods face persistent challenges in autonomous driving applications. Under severe weather conditions where visual information becomes heavily obscured, most methods struggle to extract sufficiently informative features from unlabelled data. Inconsistent loss calculations between labeled and unlabelled data further degrade practical deployment performance. The continued reliance on pixel-level annotations creates a fundamental bottleneck considering the extensive weather and lighting variations in real-world driving. These limitations underscore the urgent need for computationally efficient approaches that concurrently reduce camera and sensor system costs. A clear gap exists in current literature: existing methods fail to extract robust features from images degraded by adverse weather and struggle with inconsistent loss calculations between labeled and unlabelled data. These shortcomings hinder the development of reliable perception systems for autonomous driving [ 23 ] . To overcome these critical challenges, we present WeatherSeg —a novel universal framework for semi-supervised semantic segmentation. Our lightweight architecture seamlessly integrates with existing networks, effectively processes blurred adverse weather imagery, intelligently assigns weights, and resolves common mean-teacher training limitations. WeatherSeg provides a flexible, efficient, and robust environmental perception solution that significantly improves model generalisation, simplifies training complexity, enhances stability, and reduces hardware costs. By efficiently leveraging unlabelled data while minimising dependency on expensive annotations, our framework streamlines training processes and achieves substantial cost reduction. Our work makes three primary contributions as Fig. 1 : • Dual Teacher-Student Weight-Sharing Model (DTSWSM) : Processes fused labeled and unlabelled data streams using consistency loss training and classification pre-training. This dual-channel architecture employs separate training paths with weight sharing, enabling teacher models to effectively guide student networks and substantially improve prediction performance. • Classifier Weight Updating Attention Mechanism (CWUAM) : Sharpens distillation model predictions by updating classifier weights using preceding model outputs, achieving strong performance on unlabelled data. • Comprehensive Evaluation : Demonstrates through extensive testing across diverse conditions—including clear weather, varying fog/rain intensities, and day-night transitions—that WeatherSeg effectively handles complex visual information while significantly improving foundational model generality and adaptability. The paper organises as follows: Section II details existing semantic segmentation methods, Section III comprehensively describes our proposed framework and training methodology, Section IV presents comparative experimental results, and Section V summarises our work while outlining future directions. Fig. 1 : WeatherSeg: The whole network architecture of the dual teacher-student with classifier weight transformer. This architecture comprises two main components. The first is the Dual Teacher-Student Weight-Sharing Model (DTSWSM), which mixes and fuses dynamic vehicle data. It guides the training of shared parameters and distills primary feature information. The second component is the Classifier Weight Updating Attention Mechanism (CWUAM). Centered around a classifier attention mechanism, CWUAM sharpens the output from the first component to achieve superior semi-supervised semantic segmentation results. II Related Work Semantic segmentation is a cornerstone technology for autonomous vehicles, enabling them to achieve a detailed, pixel-level understanding of their environment [ 29 ] , [ 25 ] . While foundational deep learning models have advanced this field, significant challenges remain, particularly in label-scarce and adverse weather scenarios. This section analyses existing architectures and semi-supervised techniques, highlighting the critical gaps our proposed framework addresses. Current methodologies shaping semi-supervised segmentation include [ 10 ] , [ 18 ] , [ 17 ] , [ 24 ] , pseudo-labelling methods [ 36 ] , [ 37 ] , [ 18 ] , and entropy minimisation techniques [ 38 ] . Recent advances also explore generative modelling [ 16 ] . Techniques like View-coherent Correlation Consistency [ 10 ] maintain prediction robustness across augmented image views, while Cross-Consistency Training [ 24 ] introduces perturbations to enforce decoder consistency. Other methods [ 18 ] combine Mean Teacher networks [ 32 ] with confidence-weighted loss to refine consistency learning and reduce prediction uncertainty. Adaptive threshold adjustment [ 36 ] , [ 18 ] further improves pseudo-labelling quality throughout training. Pioneering architectures like Fully Convolutional Network (FCN) [ 31 ] , SegNet [ 1 ] , U-Net [ 8 ] , [ 30 ] , and DeepLab [ 2 ] established the basis for end-to-end semantic segmentation. FCN introduced pixel-wise prediction but suffers from high computational costs. SegNet and U-Net utilise encoder-decoder structures to refine spatial details, yet they still lose critical spatial information, struggle with occlusions, and demand significant computational resources. Researchers have attempted to optimise these models through compression and hardware acceleration [ 26 ] , [ 28 ] . However, incremental improvements often introduce new limitations. For example, Gautam et al. [ 8 ] enhanced U-Net with clustering algorithms but increased computational overhead and manual effort. Similarly, Lu et al. [ 20 ] developed an effective few-shot segmentation

Read the complete text