Deepfake Detection of Face Images based on a Convolutional Neural Network

Lukas Kroiß, Johannes Reschke · 2025

Read Deepfake Detection of Face Images based on a Convolutional Neural Network by Lukas Kroiß, Johannes Reschke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11389.txt.

Introduction

Lukas Kroiß and Johannes Reschke build a detector for deepfake portrait photos. Software that fabricates faces has become easy for anyone to use, and fake images of public figures can spread false stories fast, so a reliable way to flag them matters.

They start from a well-known image network that was already trained on millions of photos, then retrain its top layer to answer one question: real or fake? Their training set mixes many kinds of manipulation and a wide range of faces. The finished detector scored near the top on every measure they report, catching most fakes while rarely accusing real photos. The authors present it as a practical guard against portrait forgery.

Catalog record

CREATORLukas Kroiß, Johannes Reschke
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-11389

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deepfake Detection of Face Images based on a Convolutional Neural Network Lukas Kroiß, Johannes Reschke Abstract Fake News and especially deepfakes (generated, non-real image or video content) have become a serious topic over the last years. With the emergence of machine learning algorithms it is now easier than ever before to generate such fake content, even for private persons. This issue of generated fake images is especially critical in the context of politics and public figures. We want to address this conflict by building a model based on a Convolutions Neural Network in order to detect such generated and fake images showing human portraits. As a basis, we use a pre-trained ResNet-50 model due to its effectiveness in terms of classifying images. We then adopted the base model to our task of classifying a single image as authentic/real or fake by adding an fully connected output layer containing a single neuron indicating the authenticity of an image. We applied fine tuning and transfer learning to develop the model and improve its parameters. For the training process we collected the image data set ’Diverse Face Fake Dataset’ containing a wide range of different image manipulation methods and also diversity in terms of faces visible on the images. With our final model we reached the following outstanding performance metrics: precision = 0.98, recall 0.96, F1-Score = 0.97 and an area-under-curve = 0.99. I Introduction Fake news - one of the most current, most popular, but also most problematic topics worldwide. Fake news have probably existed for thousands of years, but have become increasingly important in recent times. Particularly thanks to the internet, the use and spread of fake news have grown exponentially lately. In particular young people use social media platforms on the internet to inform themselves and keeping up-to-date. This becomes especially critical in the context of politics and public figures. [ 1 ] [ 2 ] Fake news used to appear only in the form of fake texts, but is no longer limited to this single format: a new scope called deepfake can also be counted as fake news. The term is a combination of ‘deep learning’ and ‘fake’. Deepfake therefore refers to the falsification of content, usually images and videos, using generative machine learning algorithms. Since these algorithms are continuously becoming better and cheaper, the creation and use of such media is steadily increasing also by private persons. Fake news in the form of images and videos containing completely new and non-real content can be created and distributed even faster as before using deepfake technologies. [ 1 ] While machine learning tools are usually developed to create innovative solutions in the field of robotics, autonomous systems or speech models, they can also be used for creating deepfakes enabling these to become increasingly authentic and real. In images for example, certain characteristics or facial expressions of people can be changed or even entire faces can be replaced by another persons face. By doing this, deepfakes are misleading and dramatically influence current public businesses like elections worldwide. Therefore it is increasingly important, but at the same time more complicated, to reliably detect such deepfake content. [ 1 ] [ 3 ] As machine learning algorithms, especially neural networks are used to generate such images, it appears that similar technologies can be used to recognise deepfakes. To counter the misuse of deepfakes, machine learning-based tools are being developed to reliably detect generated images. However, these technologies also have limitations due to rapidly improving generation tools and limited training data, among other things. Nevertheless, many commercial machine learning tools are very promising in terms of deepfake detection. [ 3 ] [ 4 ] Current research projects focus on this field of deepfake detection providing promising results. Rössler et al. for example introduced an advanced benchmark dataset for detecting manipulated facial images and videos in [ 5 ] . The dataset supports four manipulation methods like DeepFakes (specific method for target face replacement), Face2Face (expression transfer while maintaining the identity of the person), FaceSwap (transfer of a face region or complete face to a target person) and NeuralTextures (facial re-enactment with machine learning based optimizations). Based on this benchmark, they compared different well known network architectures in terms of classifying deepfakes as authentic/real or fake. The best trained model, a XceptionNet architecture (i.e., depth wise separable convolution across all channels including shortcuts like in ResNet), achieved a binary detection accuracy of up to 99.26 % [ 5 ] . However the XceptionNet architecture is often not ideal especially for smaller dataset since they are computationally intensive and there is the risk of overfitting [ 6 ] . Unlike the traditional method to use the whole face in deepfake detection tasks, Tolosana et al. in [ 7 ] explored a method using partial face clues. This method uses specific facial parts such as eyes, nose, or mouth to distinguish between authentic and fake. For the traditional method, again, a Xception model performed best with an area-under-curve (AUC) score up to 100 %. In contrary, for the second method, the highest AUC score of 100 % was achieved as well by a Capsule Network (i.e., dynamic routing to capture spatial hierarchies and relationships in data) based on the eyes region of a face [ 7 ] . Nevertheless, these kind of networks are computationally intensive and have a complex model structure [ 8 ] . This paper is intended to make a contribution in this area of classifying deepfake images as authentic or fake in terms of a simple and at the same time accurate machine learning algorithm. We used the ResNet-50 model [ 9 ] as a basis for our model. This neural network consists of 50 parameter adoptable convolutional layers paired with residual connections every third layers. The top layer of the network is adopted to fit the input images, while the last fully connected layer is replaced by two own layers specific for this task: one flatten layer and one 1-d fully connected dense layer. The very last layer consists of a single output neuron with a sigmoid activation function. The output thus displays a single probability that an authentic image was presented and therefore serves as a classifier. Based on the Diverse Fake Face Dataset (DFFD) [ 10 ] , an AUC score of 0.9931 and a F1-score of 0.9715 were achieved by the trained model. II Methods We build our model using the ResNet-50 model [ 9 ] as baseline and adopted this to our task. For training, we applied transfer learning using the DFFD dataset. These aspects will be explained in detail in the following sections. II-A Model ResNet-50 [ 9 ] is a residual convolutional neural network model which uses an advanced model architecture to address the degradation problem occurring in training deep neural nets. Adding more layers to a neural network to build a deep net, causes the accuracy to saturate and at some point degrading rapidly. This effect is not a result of overfitting but is simply caused by the difficulty of training deep neural nets. To counteract this degradation problem, residual networks use identity shortcuts. [ 9 ] [ 11 ] These shortcuts allow a direct

Read the complete text