Semantic Style Transfer for Enhancing Animal Facial Landmark Detection
Anadil Hussein, Anna Zamansky, George Martvel · 2025
Read Semantic Style Transfer for Enhancing Animal Facial Landmark Detection by Anadil Hussein, Anna Zamansky, George Martvel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-05640.txt.
Introduction
Anadil Hussein, Anna Zamansky, and George Martvel test, in a 2025 paper, whether restyled photos can teach computers to find points on animal faces. Finding landmarks, such as eyes and nose, on pet faces feeds health and behaviour tools. Style transfer, which paints one photo's look onto another, is their tool for making more training data.
As a test case they used a cat face detector with forty-eight landmarks. Restyling cropped face images, not whole bodies, kept the structure cleaner. Swapping in restyled images raised a snag: the landmark notes no longer lined up. A supervised variant that picks style sources by landmark accuracy kept up to ninety-eight percent of the base score. Adding restyled images as extra data beat standard augmentation tricks.
Catalog record
| CREATOR | Anadil Hussein, Anna Zamansky, George Martvel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-05640
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic Style Transfer for Enhancing Animal Facial Landmark Detection Anadil Hussein, Anna Zamansky, George Martvel Abstract Neural Style Transfer (NST) is a technique for applying the visual characteristics of one image onto another while preserving structural content. Traditionally used for artistic transformations, NST has recently been adapted, e.g., for domain adaptation and data augmentation. This study investigates the use of this technique for enhancing animal facial landmark detectors training. As a case study, we use a recently introduced Ensemble Landmark Detector for 48 anatomical cat facial landmarks and the CatFLW dataset it was trained on, making three main contributions. First, we demonstrate that applying style transfer to cropped facial images rather than full-body images enhances structural consistency, improving the quality of generated images. Secondly, replacing training images with style-transferred versions raised challenges of annotation misalignment, but Supervised Style Transfer (SST) — which selects style sources based on landmark accuracy - retained up to 98% of baseline accuracy. Finally, augmenting the dataset with style-transferred images further improved robustness, outperforming traditional augmentation methods. These findings establish semantic style transfer as an effective augmentation strategy for enhancing the performance of facial landmark detection models for animals and beyond. While this study focuses on cat facial landmarks, the proposed method can be generalized to other species and landmark detection models. 1 Introduction Neural Style Transfer (NST) is a research field studying the applications of neural networks to render images in different styles by retaining the structure of the original content while adopting the visual elements of a chosen style jing2019neural ; singh2021neural . NST has numerous applications in art generation, photo, and video editing and many more v1Gayts2015ArtisticStyle ; v1JohnsonSuperResoluation2016 ; v1ParkStyle-Attentional2018 . Recent advancements in Vision Transformers (ViTs) vit have introduced a new paradigm in style transfer, applying self-attention mechanisms to capture long-range morphological dependencies in an image. Splice ViT tumanyan2022splicing introduced a novel approach to semantic visual appearance transfer, ensuring that style transformations maintain spatial structure. Unlike GAN-based models goodfellow2014generative that rely on adversarial training, Splice ViT leverages pre-trained Vision Transformers to encode content and style separately, allowing for more controlled transformations. This study explores the use of NST algorithms in a novel domain — animal affective computing. Automated analysis of facial expressions is a crucial challenge in this emerging field Broome2022 . One of the most promising approaches in this context is facial landmarks, which are well-studied for humans wu2018look ; belh13lfpw ; Le12helen and are now also being adopted for animals Martvel2024Detection ; pessanha2022facial ; hewitt2019pose ; martvel2024dogflw . Facial landmarks (keypoints/fiducial points) detection wu2019facial is extensively studied for human automated facial analysis. Landmark locations have been shown to provide crucial insights in the context of face alignment, feature extraction, facial expression recognition, head pose estimation, eye gaze tracking, and other tasks gazeestimation ; MAlEidan2020DeepLearningBasedMF ; MALEK2021406 . They have also been used for automated facial movement recognition systems FAU ; AUIP . An important advantage of landmark-based approaches is their ease of application to video data, producing time series of multiple coordinates, which can then be analyzed zhan2021key , classified sehara2021real ; ferres2022predicting ; liu2022murine , or processed for different purposes hardin2022using ; suryanto2022using . Landmark-based approaches are just beginning to be explored in the domain of animal behavior, primarily for body landmarks (e.g., labuguen2019primate ; wiltshire2023deepwild ; jeon2023deep ), while facial analysis still remains relatively underexplored. One significant challenge arises from the variety of textures, shapes, and morphological structures found across different breeds and species. This is especially true for domesticated animals, such as farm and companion ones, which exhibit significant variability in appearance and morphology vila1999phylogenetic . Another challenge in this field is the lack of large, high-quality datasets. Typically, in the human domain, datasets with facial landmarks consist of thousands of images with dozens of landmarks. The animal domain, on the other hand, severely needs landmark-related datasets and benchmarks, as highlighted in Broomé et al. broome2023going . These are just beginning to be developed for species such as cats sun20cafm , dogs liu2012dog , horses pessanha2022facial , cattle coffman2024cattleface , and sheep hewitt2019pose ; however, in most cases, the limited number of instances in the training data, the small number of landmarks, as well as lack of justification for their placement in terms of facial muscles, makes these tools inadequate for capturing the subtle facial changes necessary for emotion or pain recognition. Martvel et al. martvel2024automated recently addressed this gap for cat and dog facial analysis by introducing datasets and detector models for 48 and 46 anatomy-based facial landmarks for cats and dogs, respectively Martvel2024Detection ; martvel2024dogflw . The detector has been utilized in a number of studies, performing well in tasks requiring subtle facial analysis, such as breed, cephalic type, and pain recognition in cats martvel2024automatedA ; martvel2024automatedD . However, data scarcity still remains an issue on the way to better performance in the task of landmark detection. To address the bottlenecks of data scarcity, data augmentation techniques such as random cropping, flipping, rotation, and brightness adjustments dataAugmentation are commonly used. In this study, we explore a more sophisticated augmentation using NST, while prior work has explored semi-supervised style transfer for boosting facial landmark detection in human faces styleGAN , we apply style transfer on animals for the first time in this context, investigating both supervised and semi-supervised style transfer approaches, handling the morphological variability of animal species. In this study, we explore Neural Style Transfer as a data augmentation technique that stylistically transforms animal facial images while preserving key morphological structures crucial for landmark detection. 2 Methodology 2.1 Model For style transfer, we employ Splice ViT Tumanyan2022SpliceViT , a Vision Transformer-based approach that ensures semantic consistency during style transformations. Unlike traditional NST methods, which can disrupt spatial alignment, Splice ViT separates content and style representations, preserving facial structures critical for landmark detection. One of the challenges in applying style transfer to landmark-based models is annotation misalignment—where stylistic transformations alter the image while retaining the original landmark coordinates. To mitigate this, we incorporate Supervised Style Transfer (SST), selecting style reference images with minimal landmark displacement, ensuring that transformations do not compromise spatial accuracy. 2.2 Dataset and Preprocessing This study utilizes the Cat Facial Landmarks in the Wild (CatFLW) dataset Martvel2023CatFLW , which consists of 2,091 annotated feline facial images with 48 anatomical facial landmarks. The dataset provides a diverse representation of cat faces, enabling robust training for landmark detection models. To maintain structural consistency in style transfer, we preprocess the dataset by isolating the cat’s face region from the full image. Prior studies suggest that applying style transfer to full-body images can introduce background artifacts, feature distortions, and misalignment of facial landmarks Martvel2024Analysis . To mitigate these issues, we compare two preprocessing strategies: 1. Full-Body Style Transfer (FB-ST): Style transfer is applied to the entire image, preserving background context. 2. Cropped-Face Style Transfer (CF-ST): The cat’s face is extracted before style transfer to minimize distortions. We evaluated which preprocessing method better preserves facial structure by testing 30 random image pairs using both FB-ST and CF-ST for style transfer. The structural preservation was evaluated using two metrics: the Intersection over Union (IoU) between the original and style-transferred segmentation masks, and the Splice ViT loss function (