Occlusion-Aware Face Recognition via Adaptive Local Feature Fusion and Identity-Guided Contrastive Learning Authors: Kexin Zhu, Guoqing Ma Published: 2026 ABSTRACT Partial occlusion can substantially impair the accuracy and stability of face recognition systems. Although existing methods perform well on unobstructed face images, their performance often degrades under partial occlusion, because occluded regions may obscure discriminative identity cues and introduce noise into feature representations. To address this issue, this paper proposes an occlusion-aware face recognition framework that integrates lightweight feature extraction, local-region reliability modeling, adaptive feature fusion, and joint loss optimization. Specifically, the face is divided into three sub-regions according to common occlusion patterns, and an MLP-based module is used to estimate the reliability of each region. The estimated reliability weights are then used to adaptively fuse local features, thereby emphasizing visible and discriminative regions. In addition, a joint loss combining ArcFace and InfoNCE is constructed to enhance inter-class separability and intra-class feature consistency. Experimental results under masks, hats, sunglasses, and random occlusion conditions show that the proposed method achieves a recognition accuracy of 92.3%. Compared with ArcFace, CurricularFace, and AdaFace, the proposed method improves accuracy by 9.9%, 6.5%, and 4.1%, respectively. In addition, the FAR is reduced by 5.8%, 4.9%, and 3.7%, respectively, while the FRR is reduced by 2.2%, 6.5%, and 1.3%, respectively. These results demonstrate that the proposed framework effectively enhances the robustness of face recognition under partial occlusion. 1. Introduction Partial occlusion can substantially impair the accuracy and stability of face recognition systems. Although existing methods perform well on unobstructed face images, their performance often degrades under partial occlusion, because occluded regions may obscure discriminative identity cues and introduce noise into feature representations. To address this issue, this paper proposes an occlusion-aware face recognition framework that integrates lightweight feature extraction, local-region reliability modeling, adaptive feature fusion, and joint loss optimization. Specifically, the face image is horizontally divided into three sub-regions according to common occlusion patterns. LA-ResNet50 is then used to extract features from each facial region, and an MLP-based module is employed to estimate the reliability of each region. The estimated reliability weights are subsequently used to adaptively fuse local features, thereby emphasizing visible and discriminative regions. In addition, a joint loss combining ArcFace and InfoNCE is constructed to enhance inter-class separability and intra-class feature consistency. Experimental results under mask, hat, and sunglasses occlusion conditions show that the proposed method achieves a recognition accuracy of 92.3%. Compared with ArcFace, CurricularFace, and AdaFace, the proposed method improves accuracy by 9.9%, 6.5%, and 4.1%, respectively. In addition, the FAR is reduced by 5.8%, 4.9%, and 3.7%, respectively, while the FRR is reduced by 2.2%, 6.5%, and 1.3%, respectively. These results demonstrate that the proposed framework effectively enhances the robustness of face recognition under partial occlusion. 2. Related Work 2.1. Occluded Face Recognition Face recognition is a fundamental computer vision task that aims to determine a person’s identity from facial images. With the development of deep learning, deep face recognition methods based on discriminative feature embeddings have achieved significant progress and have become a mainstream solution for identity recognition tasks [1]. Compared with traditional hand-crafted feature methods, deep learning-based approaches show stronger capabilities in nonlinear feature representation, high-dimensional data processing, and discriminative feature extraction. However, most conventional face recognition models are designed for complete or near-frontal facial images. When these models are directly applied to partially occluded facial images, their recognition performance may drop significantly. In practical scenarios, facial images are often affected by masks, sunglasses, hats, hands, and other accessories. These occlusions may cover key identity-related regions and introduce irrelevant visual interference, resulting in the loss of effective identity information and reduced recognition accuracy and stability [2]. Existing studies on occluded face recognition mainly focus on facial feature restoration and attention-enhanced recognition. Facial feature restoration methods attempt to recover missing or corrupted facial regions before recognition, while attention-enhanced methods improve the robustness of feature representation by guiding the model to focus on more discriminative facial regions. 2.2. Facial Feature Restoration Methods Facial feature restoration methods aim to reconstruct missing or corrupted facial regions by generating semantically reasonable and contextually coherent facial details. These methods usually involve facial degradation modeling and deep learning-based image reconstruction [3]. Existing restoration-based methods attempt to reconstruct or recover occluded facial regions using generative facial priors, image inpainting, or de-occlusion strategies [4,5,6]. Some studies further combine facial restoration, identity information preservation, and recognition tasks to improve the identity consistency of restored facial images [7]. Although restoration-based methods can improve the visual completeness of occluded faces, they still have several limitations. First, these methods usually require complex network structures, paired high-quality and low-quality facial images, and accurate identity annotations, which increases the difficulty of data acquisition and model training. Second, the reconstructed facial details may not always correspond to the true identity information of the original face. When large occlusions occur, the generated regions may introduce artificial identity bias, which can further affect the subsequent recognition process. Therefore, when the missing regions contain important identity-related information, restoration-based methods may suffer from insufficient reliability. 2.3. Attention-Enhanced Face Recognition Methods Attention-enhanced face recognition methods introduce attention mechanisms into deep neural networks to guide the model to focus on discriminative and reliable facial regions. Existing studies commonly use channel attention, spatial attention, or self-attention mechanisms in Transformer-based architectures to model global facial structures and local facial details [8,9,10,11,12,13]. These methods can adaptively enhance useful facial features and suppress occlusion interference, thereby improving recognition robustness in complex scenarios [14]. In addition, attention-guided pyramidal feature learning has also been explored in fine-grained image recognition to extract discriminative multi-level features from limited samples. This indicates that attention mechanisms combined with pyramidal feature representation can effectively enhance fine-grained visual recognition. However, such methods are mainly designed for general fine-grained image recognition tasks and do not explicitly model the reliability of local facial regions under occlusion conditions. Most attention-based methods mainly enhance feature responses from a global or appearance-driven perspective. They usually rely on visual saliency or feature activation strength to assign attention weights, but highly activated regions are not necessarily reliable for identity recognition. For example, occluding objects such as masks, sunglasses, or hats may also produce strong feature responses, causing the model to encode occlusion noise together with valid identity information. In addition, many attention-based methods do not explicitly evaluate the reliability of different local facial regions under different occlusion conditions. As a result, unreliable occluded regions may still contribute to the final feature representation, while discriminative non-occluded regions may not be sufficiently emphasized. 2.4. Limitations of Existing Methods In addition to restoration-based and attention-enhanced methods, some recent studies have attempted to improve face recognition performance by optimizing network architectures and training strategies for feature extraction [15,16,17,18]. Transformer-based recognition frameworks have also been applied to other visual recognition tasks, such as structural damage identification, showing their ability to learn discriminative representations under complex conditions [19,20]. Although these face recognition methods have improved recognition performance to some extent, many of them are still designed based on complete or ideal facial samples. In summary, existing occluded face recognition methods have improved recognition robustness from different perspectives, such as facial feature restoration, attention enhancement, and feature representation optimization. However, restoration-based methods may introduce identity bias when reconstructing missing facial regions, while attention-based methods usually rely on global or appearance-driven feature responses and lack explicit modeling of local-region reliability. More specifically, the main limitation lies in the lack of an identity-aware criterion for judging whether a local facial region is truly reliable for recognition. Existing methods may assign high responses to visually salient occluders, such as masks or sunglasses, but these regions do not necessarily contain valid identity information. As a result, local regions with strong activation may still introduce occlusion noise into the final representation, while truly discriminative visible regions may not be sufficiently emphasized. In addition, existing methods often struggle to adaptively determine the effectiveness of each local region for identity representation under different occlusion positions and degrees, and have difficulty distinguishing highly activated occluded regions from truly reliable identity-related regions. Although previous studies have explored local feature learning and identity-preserving constraints, how to use identity-consistency constraints to guide local-region reliability modeling and achieve adaptive fusion of reliable local features under different occlusion conditions still requires further investigation. To address these limitations, this study introduces an identity-guided local-region reliability modeling strategy, which adaptively estimates the reliability of different facial regions and fuses reliable local features under partial occlusion. 3. Proposed Methods 3.1. Overall Framework As shown in Figure 1, different colored boxes represent distinct functional modules: yellow indicates the feature extraction network (LA-ResNet50) [21], blue denotes the feature fusion process, and green represents the identity recognition process. The proposed occlusion-aware face recognition framework consists of three core modules: feature extraction, feature fusion, and matching recognition. First, three local image patches are generated by horizontally partitioning the aligned and resized face image: the upper, middle, and lower facial regions. Subsequently, the feature extraction module uses LA-ResNet50 to separately extract the identity features of each local region. Next, the feature fusion module predicts the original contribution score of each local region through a two-layer MLP and uses Softmax to generate normalized weights, thereby achieving adaptive weighted fusion of local features. Finally, the fused feature is compared with the facial feature vectors in the database through similarity calculation to complete identity authentication. The following subsections provide a detailed description of each component of the framework. 3.2. Local Facial Region Generation Given a resized face image I∈RH×W×C, local facial regions are generated by uniformly dividing the image along the height dimension. Specifically, the image is divided into three horizontal local patches, denoted as p1, p2, and p3, corresponding to the upper, middle, and lower facial regions, respectively. Let h=[H/3]. The first two patches have a height of h, while the last patch covers the remaining pixels to preserve the complete image region. The three patches are defined as(1)p1=[0:h,:,:], p2=[h:2h,:,:], p3=[2h:H,:,:], where h=H/3 patches are then used as the inputs to the subsequent feature extraction network. 3.3. Lightweight Feature Extraction with L-BTNK1 To reduce computational complexity while preserving discriminative facial representations, we construct a lightweight feature extraction network named LA-ResNet50 based on the original ResNet50. Compared with ResNet50, LA-ResNet50 retains the stage-wise backbone architecture but introduces two main modifications. First, the original BTNK1 blocks in Stage2 and Stage3 are replaced with the proposed L-BTNK1 modules to enhance local facial feature extraction under occlusion. Second, two repeated BTNK2 blocks are removed from Stage4 to reduce the network depth, parameter number, and computational complexity. The other stages are kept consistent with the original ResNet50 to preserve the hierarchical representation ability of the backbone network. For each local facial patch pi, LA-ResNet50 is used as a shared feature extractor to obtain its D-dimensional feature representation:(2)fi=E(pi), fi∈RD where E(⋅) denotes the proposed LA-ResNet50 and D denotes the dimension of the extracted feature vector. As shown in Figure 2, LA-ResNet50 retains the overall stage-wise architecture of the original ResNet50, including the initial convolutional layer, max-pooling layer, and the subsequent hierarchical feature extraction stages. The input face image first passes through a 7 × 7 convolutional layer, followed by batch normalization, ReLU activation, and max pooling, which reduces the spatial resolution while extracting low-level texture and edge information. The resulting feature maps are then progressively processed by Stage2 to Stage5 to obtain increasingly abstract and discriminative facial representations. Compared with the original ResNet50, the main modification of LA-ResNet50 lies in the replacement and simplification of bottleneck blocks. Specifically, L-BTNK1 is introduced into Stage2 and Stage3 to enhance local feature representation. Since these stages still retain relatively rich spatial information, introducing L-BTNK1 at this level helps the network capture fine-grained facial details, such as local textures around the eyes, nose, and mouth, which are important for recognizing partially occluded faces. In L-BTNK1, the local pyramid structure combines regular convolution and dilated convolution branches, enabling the network to capture both local details and a larger receptive field without substantially increasing the number of parameters. In addition, two repeated BTNK2 blocks are removed from Stage4 to achieve lightweight feature extraction. Each BTNK2 block contains multiple convolutional operations, including 1×1 convolution, 3×3 depthwise convolution, and a residual connection. Therefore, removing repeated BTNK2 blocks directly reduces the network depth, the number of convolutional operations, and the overall computational cost. Since Stage4 already contains high-level semantic information, reducing part of the repeated BTNK2 blocks can decrease redundant feature computation while preserving the hierarchical representation ability of the backbone network. After the hierarchical feature extraction process, the feature maps are further processed by Stage5 and global average pooling to generate the final feature vector. In this way, LA-ResNet50 maintains the multi-stage representation learning capability of ResNet50 while improving local feature enhancement and reducing computational complexity. This design enables the backbone network to extract more robust identity-related features from visible facial regions under partial occlusion conditions. As shown in Figure 3, the proposed L-BTNK1 module is designed based on the original bottleneck block of ResNet50. Given an input feature map X∈RC×H×W, 1×1 convolution is first used to reduce the channel dimension. Then, the reduced feature is fed into the Local Pyramid Module (LPM). In the LPM, two parallel convolutionbranches are adopted. One branch uses a 3×3 depthwise convolution to capture fine-grained local facial details, while the other branch uses a 3×3 depthwise dilated convolution [22] with a dilation rate of 2 to enlarge the receptive field and extract contextual information. The outputs of the two branches are concatenated along the channel dimension and then fused by a shared 1×1 pointwise convolution. After multi-scale local feature fusion, a Squeeze-and-Excitation (SE) module is introduced to recalibrate channel-wise feature responses. SE is a classic channel attention module widely used in convolutional neural networks. It adaptively models the importance of different feature channels, thereby enhancing informative channels and suppressing less useful ones. In the proposed L-BTNK1 module, this helps improve the discriminative representation of local facial features while maintaining a lightweight structure. Finally, a 1×1 convolution is used to restore the channel dimension, and the output is combined with the shortcut branch through residual connection and ReLU activation. In this way, L-BTNK1 can enhance local feature representation while maintaining a lightweight structure. To further illustrate the role of the dilated convolution branch in the Local Pyramid Module (LPM), Figure 4 compares the receptive fields of regular 3 × 3 convolution and dilated 3 × 3 convolution with a dilation rate of 2. As shown in Figure 4, compared with regular 3×3 convolution, dilated 3×3 convolution with a dilation rate of 2 enlarges the effective receptive field by introducing intervals between adjacent kernel sampling positions. Meanwhile, the number of learnable kernel parameters remains unchanged. Therefore, the dilated convolution branch in the LPM can capture broader contextual information without increasing the number of convolutional parameters, which helps improve local feature representation under partial occlusion. 3.4. Learnable Occlusion-Aware Feature Fusion A lightweight contribution estimation module is designed to assign adaptive reliability weights to different facial patches. Specifically, each local feature vector extracted by LA-ResNet50 is fed into a two-layer MLP with shared parameters to obtain its corresponding original contribution score. Then, Softmax normalization is applied to all patch-level scores to generate reliability weights whose sum equals 1. These weights can adaptively change according to the input features, dynamically reflecting the discriminative value of each local region in the current sample. During training, the module does not rely on explicit occlusion labels, but automatically learns through end-to-end backpropagation with the joint loss. As a result, higher weights are assigned to unoccluded and more discriminative facial patches, while lower weights are given to regions with severe occlusion or noise interference. Specifically, for the i-th facial patch, its feature representation is denoted as fi∈RD. The contribution estimation module adopts a two-layer MLP to map fi to a scalar contribution score. The first fully connected layer performs dimensionality transformation, followed by the ReLU activation function to introduce nonlinear representation capability. Then, the second fully connected layer maps the transformed feature to a scalar value representing the original contribution score of the patch. This process can be expressed as(3)Si=W2σ(W1fi+b1)+b2, where Si denotes the original contribution score of the i-th facial patch, W1 and W2 are learnable weight matrices, b1 and b2 are bias terms, and σ(⋅) represents the ReLU activation function. To ensure the relative comparability of contribution scores among different facial patches and constrain the weights to a normalized range, Softmax normalization is applied to the original contribution scores {Si}i=13 The final reliability weight of the i-th patch is defined as(4)wi=exp(Si)∑j=13exp(Sj), where Si denotes the original contribution score of the i-th facial patch, and exp(⋅) denotes the exponential function. The resulting weights satisfy wi∈(0,1) and ∑i=13wi=1, and can be directly used for subsequent weighted feature fusion. The final fused feature representation is obtained by weighted aggregation of the local feature vectors:(5)Ffused=∑i=13wifi During training, the contribution estimation module does not rely on any explicit occlusion annotations. Instead, its parameters are learned jointly with the feature extraction network in an end-to-end optimization manner. Since the InfoNCE contrastive learning loss [23] is applied to the fused representation, the gradients can be backpropagated through Ffused to the reliability weights wi and the learnable parameters of the MLP, including (W1,b1,W2,b2). In this way, the module is encouraged to automatically learn reliable weight allocation under different occlusion and interference conditions, thereby achieving occlusion-aware feature fusion and stable feature representation. 3.5. Training and Joint Optimization To enhance both the identity discrimination ability and occlusion robustness of the model, this study constructs a joint optimization objective consisting of ArcFace loss and InfoNCE loss. The joint loss is defined as follows:(6)Ltotal=LArcFace+λLInfoNCE, Here, ArcFace enhances intra-class compactness and inter-class separability by introducing an angular margin constraint, thereby improving identity classification accuracy. However, in locally occluded face recognition scenarios, the reliability of facial features may vary significantly across samples. Severe occlusion can reduce the consistency of identity features and make the angular distribution of samples from the same identity more dispersed. Therefore, directly using a fixed and relatively large angular margin may impose overly strong constraints on low-quality occluded samples and lead to unstable training and noisy gradients [24,25]. For the ground-truth class yi, ArcFace replaces the target logit from cosθyi with cos(θyi+m) and introduces a scaling factor (s) to obtain the softmax cross-entropy loss. To address the instability caused by fixed-margin optimization under occlusion, a weight-concentration factor q is introduced, and the angular margin is reformulated as m(q), resulting in the adaptive ArcFace loss:(7)LArcFace=−1N∑i=1Nlog(exp(s⋅cos(θyi+m(q)))exp(s⋅cos(θyi+m(q)))+∑j≠yiexp(s⋅cosθj)) The factor q is derived from the local-region reliability weights wi=1K, where ∑i=1Kwi=1, wi∈(0,1). In this study, q is defined as the sum of squared local-region reliability weights:(8)q=∑i=1Kwi2 According to q, a linear mapping is used to construct the adaptive angular margin:(9)m(q)=mmax−(mmax−mmin)q, 0