MoDE: Mixture of Diffusion Experts for Any Occluded Face Recognition
Qiannan Fan 1,2 , Zhuoyang Li 2 , Jitong Li 2† , Chenyang Cao 2∗ 1 State Grid Tianjin Economic Research Institute, Tianjin, China 2 College of Intelligence and Computing, Tianjin University, Tianjin, China [email protected], [email protected], [email protected], [email protected] · 2025
Read MoDE: Mixture of Diffusion Experts for Any Occluded Face Recognition by Qiannan Fan 1,2 , Zhuoyang Li 2 , Jitong Li 2† , Chenyang Cao 2∗ 1 State Grid Tianjin Economic Research Institute, Tianjin, China 2 College of Intelligence and Computing, Tianjin University, Tianjin, China [email protected], [email protected], [email protected], [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04306.txt.
Introduction
Qiannan Fan, Zhuoyang Li, Jitong Li, and Chenyang Cao build a face reader in 2025 that works when part of the face is covered. Masks became a daily sight in epidemic years, and most face systems fail on covered faces of new types or depths. Their system uses a team of image builders, each guessing one full face from the covered one.
Since rebuilt faces vary by chance, a gating net scores how much each guess helps pin the person's identity. The guesses are then merged at the choice level. The module can plug into most face systems already in use, they note. Tests on three public sets and two wild sets beat past methods across cover types. Covered-face reading matters for phones, gates, and clinics alike.
Catalog record
| CREATOR | Qiannan Fan 1,2 , Zhuoyang Li 2 , Jitong Li 2† , Chenyang Cao 2∗ 1 State Grid Tianjin Economic Research Institute, Tianjin, China 2 College of Intelligence and Computing, Tianjin University, Tianjin, China [email protected], [email protected], [email protected], [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04306
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MoDE: Mixture of Diffusion Experts for Any Occluded Face Recognition Qiannan Fan 1,2 , Zhuoyang Li 2 , Jitong Li 2† , Chenyang Cao 2∗ 1 State Grid Tianjin Economic Research Institute, Tianjin, China 2 College of Intelligence and Computing, Tianjin University, Tianjin, China [email protected], [email protected], [email protected], [email protected] Abstract With the continuous impact of epidemics, people have become accustomed to wearing masks. However, most current occluded face recognition (OFR) algorithms lack prior knowledge of occlusions, resulting in poor performance when dealing with occluded faces of varying types and severity in reality. Recognizing occluded faces is still a significant challenge, which greatly affects the convenience of people’s daily lives. In this paper, we propose an identity-gated mixture of diffusion experts (MoDE) for OFR. Each diffusion-based generative expert estimates one possible complete image for occluded faces. Considering the random sampling process of the diffusion model, which introduces inevitable differences and variations between the inpainted faces and the real ones. To ensemble effective information from multi-reconstructed faces, we introduce an identity-gating network to evaluate the contribution of each reconstructed face to the identity and adaptively integrate the predictions in the decision space. Moreover, our MoDE is a plug-and-play module for most existing face recognition models. Extensive experiments on three public face datasets and two datasets in the wild validate our advanced performance for various occlusions in comparison with the competing methods. † † footnotetext: † Work done during internship at Tianjin University. † † footnotetext: * Corresponding author. I Introduction Face recognition is ubiquitous and has been widely applied in various applications [ 1 ] , such as security, payment, and social media. Recent face recognition algorithms [ 2 , 3 ] , such as FaceNet [ 4 , 5 ] , CosFace [ 6 ] , and ArcFace [ 7 ] , have shown remarkable performance in unobstructed face recognition. However, many practical scenarios may cause the face to be obscured. In particular, the frequent flu epidemics have led to more people getting used to wearing masks to protect themselves and others, bringing new challenges to face recognition, which significantly impacts the convenience of daily security. Moreover, the uncertainty of occluded face regions and the diversity of occlusion types ( e.g. , accessories, glasses, hats, and masks) require more robust and generalized face recognition techniques [ 8 ] . The occluded face parts usually result in discriminative information loss and feature changes, leading to a significant decline in the recognition accuracy of most traditional face recognition models. Fig. 1 : To demonstrate the adaptability of our method for multi-occluded face recognition, we visualize some face images from the LFW dataset with six different face occlusions, each image is repainted below. The right part shows the accuracy comparison between the baseline and our method. In recent years, some studies have emerged to improve occluded face recognition accuracy. These methods can be mainly grouped into three categories [ 9 ] . i) Occlusion robust feature extraction (ORFE) aims to extract features that are less affected by occlusions, such as Local Binary Patterns (LBP) [ 10 , 11 ] and Elastic Bunch Graph Matching (EBGM) [ 12 ] . However, most of these methods rely on handcrafted features and require prior knowledge, which limits their effectiveness. ii) Occlusion-aware face recognition (OAFR) discards occluded parts and only uses visible face parts that qualify for face recognition. This approach requires detecting the occluded parts, which can be challenging in the case of irregular occlusions. iii) Occlusion recovery-based face recognition (ORecFR) uses a reconstruction model to repair occluded faces for the demands of conventional face recognition systems. However, these models may not generalize well to occlusions that are not present in the training set, which could result in unsatisfied results [ 13 ] . Furthermore, while the reconstructed images may present visually convincing, they might be tangled in poor authenticity and identity ambiguity. In this paper, we propose an identity-gated mixture of diffusion experts (MoDE) framework for occluded face recognition. By integrating multiple diffusion experts, we obtain identity-preserving reconstructed face images while improving recognition accuracy. Specifically, the MoDE utilizes diffusion experts to generate reconstructed images for each occluded face, regardless of the type of occlusions. The diffusion model endows diverse potential visual appearances for the repaint results that provide both effective and redundant information for identification. To fully exploit the identity information from various reconstructed images, we further design an identity gating (ID-Gate) network that estimates the authenticity of multiple input images and outputs the corresponding contribution to the identification of the similarity matrix. To this end, MoDE exhibits excellent performance for multiple occluded face recognition, Fig. 1 illustrates it. Furthermore, the adaptively selected results of MoDE serve as valuable references for evaluating the identity consistency of the reconstructed images. We have performed experiments on various datasets to demonstrate the superiority against competing methods. Our main contributions can be summarized in three aspects: 1) We propose a dynamic occluded face recognition framework, which integrates multiple generative experts in the decision space through ID-Gate and subtly leverages comprehensive identity information produced by them. 2) We introduce the diffusion model to reconstruct occluded faces with high image quality and diversity. Our approach is capable of identifying various types of occlusions, even in extreme conditions. To our knowledge, our work is the first to introduce the diffusion model to occluded face recognition. 3) Our MoDE is a flexible plug-and-play module that can be seamlessly combined with most existing face recognition models and achieve stable improvements. Extensive experiments validate our superiority. II Related works General Face Recognition have achieved impressive results and have focused primarily on designing novel network architectures [ 14 , 15 ] and loss functions [ 16 , 17 ] . The early models relied on multiple deep convolutional neural networks to extract facial features, which were later fused together [ 18 , 19 ] . However, the use of a single network to extract features has now dominated mainly [ 4 ] . Meanwhile, the softmax loss for multiclass classification is widely used and significantly boosted by angle-based techniques [ 20 , 6 , 7 ] . Fig. 2 : The framework of our MoDE. Firstly, MoDE reconstructs the occluded image and produces n repainted images. Then, it extracts the features from the repainted images and the original occluded image, which are then inputted into ID-Gate to output the weight vector. Finally, we calculate the similarity matrix of each image and multiply it with the weight vector to generate a weighted similarity matrix, thereby obtaining the recognition result. Occluded Face Recognition has become a persistent challenge in computer vision [ 9 ] . Existing methods can be broadly categorized into robust occlusion feature extraction (ORFE), occlusion-aware face recognition (OAFR), and occlusion-recovery-based face recognition (ORecFR). ORFE aims to extract features from an image or video that are robust to occlusions [ 21 , 10 , 11 ] . However, the accuracy of these methods is limited by the limited discriminability of feature representation [ 22 ] . OAFR uses visible facial parts alone for recognition to alleviate the occlusion effect [ 23 , 24 , 25 ] . They mainly exploit limited information from partial faces. ORecFR [ 26