AHAN: Asymmetric Hierarchical Attention Network for Identical Twin Face Verification

Hoang-Nhat Nguyen · 2026

Read AHAN: Asymmetric Hierarchical Attention Network for Identical Twin Face Verification by Hoang-Nhat Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-21503.txt.

Introduction

Hoang-Nhat Nguyen alone authors a paper on identical twin face checks, an extreme fine-grained task. Standard face matching tops 99 percent on normal benchmarks. But scores drop to about 89 percent when telling twins apart, showing weak spots in ID security systems.

The hard part lies in learning cues that capture subtle, non-genetic changes unique to each person. The proposed Asymmetric Hierarchical Attention Network, or AHAN, studies faces at several scales. One module handles meaning regions of the face at best sizes. A second module matches left and right facial halves for unique uneven patterns. A training-only trick uses each subject's own twin as the hardest decoy. On the ND_TWIN dataset, AHAN reaches about 92 percent twin check scores, about 3 points above leading methods.

Catalog record

CREATORHoang-Nhat Nguyen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-21503

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AHAN: Asymmetric Hierarchical Attention Network for Identical Twin Face Verification Hoang-Nhat Nguyen Abstract Identical twin face verification represents an extreme fine-grained recognition challenge where even state-of-the-art systems fail due to overwhelming genetic similarity. Current face recognition methods achieve over 99.8% accuracy on standard benchmarks but drop dramatically to 88.9% when distinguishing identical twins, exposing critical vulnerabilities in biometric security systems. The difficulty lies in learning features that capture subtle, non-genetic variations that uniquely identify individuals. We propose the Asymmetric Hierarchical Attention Network (AHAN), a novel architecture specifically designed for this challenge through multi-granularity facial analysis. AHAN introduces a Hierarchical Cross-Attention (HCA) module that performs multi-scale analysis on semantic facial regions, enabling specialized processing at optimal resolutions. We further propose a Facial Asymmetry Attention Module (FAAM) that learns unique biometric signatures by computing cross-attention between left and right facial halves, capturing subtle asymmetric patterns that differ even between twins. To ensure the network learns truly individuating features, we introduce Twin-Aware Pair-Wise Cross-Attention (TA-PWCA), a training-only regularization strategy that uses each subject’s own twin as the hardest possible distractor. Extensive experiments on the ND_TWIN dataset demonstrate that AHAN achieves 92.3% twin verification accuracy, representing a 3.4% improvement over state-of-the-art methods. Introduction Consider a scenario where an unauthorized individual attempts to breach a high-security facility by exploiting their genetic similarity to an authorized twin. This represents a critical vulnerability in modern biometric systems with profound implications for national security and financial institutions. Deep learning has propelled face recognition to remarkable accuracy levels, with state-of-the-art systems achieving over 99.8% accuracy on standard benchmarks like LFW. However, these same systems experience a dramatic performance drop to approximately 88.9% when distinguishing between identical twins, revealing a fundamental limitation that could be exploited to bypass biometric authentication. Figure 1: Visualization of the generated attention map. The core challenge stems from twins sharing nearly 100% of their DNA, resulting in remarkably similar facial bone structure, skin texture, and overall appearance. Standard face recognition models excel at capturing global facial features like overall shape and proportions, but are easily confounded because these dominant features are nearly identical between twins. The truly identifying information lies in subtle, non-genetic characteristics: precise mole locations, unique patterns of fine lines, minor scars, slight asymmetries in facial structure, or minute differences developed over time due to environmental factors. For success in this extreme fine-grained task, models cannot rely on holistic appearance but must actively seek out and prioritize these minute, stable, highly localized differences while ignoring overwhelming genetic similarities. Existing approaches are fundamentally ill-equipped for this challenge. State-of-the-art face recognition methods, while powerful for general populations, primarily focus on maximizing inter-class distance across diverse individuals where genetic differences create substantial feature variations. These methods lack specialized mechanisms to resolve the near-zero distance between twin identities. Conversely, Fine-Grained Visual Categorization (FGVC) methods like DCAL ( 25 ) demonstrate expertise in finding subtle visual cues for distinguishing similar categories, but are general-purpose by design and lack the specific architectural priors needed for structured facial analysis. To bridge this gap, we propose the Asymmetric Hierarchical Attention Network (AHAN), a novel framework that synergizes face recognition and fine-grained classification strengths while introducing domain-specific innovations. AHAN deconstructs twin verification into multi-granular analysis focusing precisely on features that differentiate genetically identical individuals. Our approach recognizes that different discriminative information exists at different facial locations and scales, requiring specialized analytical strategies. The key insight is that successful twin discrimination requires simultaneous analysis at three complementary levels: global facial structure (for context), local part-based features (for fine-grained details), and facial asymmetry patterns (for unique biometric signatures). Our primary contributions are: (1) AHAN, a novel multi-stream architecture that analyzes faces at global, local, and asymmetric levels simultaneously; (2) Two new architectural modules: Hierarchical Cross-Attention (HCA) for multi-scale, part-based facial analysis and Facial Asymmetry Attention Module (FAAM) for unique asymmetry signatures; (3) Twin-Aware Pair-Wise Cross-Attention (TA-PWCA), a targeted training regularizer using each subject’s twin as the hardest distractor; (4) New state-of-the-art on ND_TWIN benchmark with 92.3% twin verification accuracy—a 3.4 percentage point improvement over existing methods. Related Work Highly Similar Faces Recognition Deep face recognition has achieved significant progress through CNNs and large-scale datasets, making systems widely deployed in real-world applications. However, distinguishing between identical twins or highly similar faces remains a major challenge due to extreme similarity in facial features that often leads to higher error rates compared to general face recognition tasks. Identical twins present the hardest cases for face recognition systems, as their facial features are nearly indistinguishable even for advanced deep learning models ( 15 ; 8 ; 12 ) . Studies consistently show that recognition accuracy drops significantly when distinguishing twins compared to unrelated individuals, regardless of lighting, expression, or temporal variations ( 14 ; 12 ; 17 ) . Conventional facial features and common deep learning embeddings often fail to capture the subtle differences needed to reliably separate twins ( 11 ; 10 ; 12 ) . While specialized feature extraction, hybrid models, and 3D data can improve accuracy, no method fully solves the problem in all conditions. This remains an active research area, especially for security and biometric applications where reliable twin discrimination is critical. Fine-Grained Visual Categorization FGVC aims to distinguish between visually similar subcategories by identifying subtle, discriminative features that differentiate closely related classes ( 1 ; 22 ; 20 ) . Recent advances focus on automatically locating discriminative object parts without explicit annotations, with attention mechanisms becoming central to highlighting informative image regions. Techniques such as cascaded attention and gradual attention mechanisms allow models to progressively refine focus on discriminative regions, improving classification by extracting subtle features ( 21 ; 5 ) . Methods that localize key parts using only image-level labels avoid costly part annotations while merging potential parts or using cross-layer interactions to enhance discriminative power ( 23 ; 19 ) . The DCAL framework ( 25 ) represents a notable Vision Transformer-based contribution, introducing Global-Local Cross-Attention (GLCA) and Pair-Wise Cross-Attention (PWCA). GLCA addresses the limitation that standard self-attention treats all regions equally by identifying high-response regions through attention rollout and computing cross-attention between local queries and global key-value pairs. PWCA serves as training regularization using random image pairs as distractors, forcing networks to learn more robust features. While DCAL demonstrates effectiveness for bird species and vehicle recognition, its design assumptions differ from twin verification. DCAL’s attention rollout works for clearly defined object parts but may not capture subtle facial differences. Similarly, DCAL’s random pair regularization creates moderate difficulty suitable for general fine-grained classification but insufficient for extreme twin discrimination. Our work draws architectural inspiration from DCAL’s cross-attention concepts but fundamentally re-engineers these components for facial analysis where discriminative information is more subtle and structurally constrained. Proposed Approach Overview In this work, we propose the Asymmetric Hierarchical Attention Network (AHAN) to address the extreme challenge of identical twin face verification. The key insight is that distinguishing twins requires analyzing faces at multiple complementary granularities, as different types of discriminative information exist at different scales and locations. AHAN employs a multi-stream architecture built upon a Vision Transformer ( 3 ) backbone that simultaneously captures three types of facial representations. The first stream uses standard self-attention to learn global facial structure and overall appearance.

Read the complete text