Current address: North Carolina Agricultural and Technical State University, 1601 E Market St, Greensboro, North Carolina 27411, USA
‡
These authors contributed equally to this work.
Version 1
: Received: 16 February 2023 / Approved: 17 February 2023 / Online: 17 February 2023 (06:51:37 CET)
How to cite:
Johnson, D.; Yuan, X.; Roy, K. Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints2023, 2023020299. https://doi.org/10.20944/preprints202302.0299.v1
Johnson, D.; Yuan, X.; Roy, K. Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints 2023, 2023020299. https://doi.org/10.20944/preprints202302.0299.v1
Johnson, D.; Yuan, X.; Roy, K. Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints2023, 2023020299. https://doi.org/10.20944/preprints202302.0299.v1
APA Style
Johnson, D., Yuan, X., & Roy, K. (2023). Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints. https://doi.org/10.20944/preprints202302.0299.v1
Chicago/Turabian Style
Johnson, D., Xiaohonh Yuan and Kaushik Roy. 2023 "Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data" Preprints. https://doi.org/10.20944/preprints202302.0299.v1
Abstract
Deepfakes are manipulated or altered images, or video, that are created using deep learning models with high levels of photorealism. The two popular methods of producing a deepfake are based on either convolutional neural networks (CNN), or autoencoders. Deepfakes created using CNN comparatively show higher qualities of realism, yet oftentimes leave artifacts and distortions in the generated media that can be detected using machine learning and deep learning algorithms. In recent years, there has been an influx of periocular image and video data because of the increase usage of face masks. By wearing masks, much of what is used for facial recognition is hidden, leaving only the periocular region visible to an observer. This loss of vital information leads to easier misidentification of media, allowing deepfakes to less likely be identified as fake. In this work, feature extraction methods, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), and CNN, are used to train an ensemble deep learning model to detect deepfakes in videos on a frame-by-frame level based on the periocular region. Our proposed model is able to distinguish original and manipulated images with accuracies around 98.9 percent, which is an improvement to previous works by combining SIFT and HOG for deepfake detection in convolutional neural networks.
Keywords
deepfake detection; CNN; deep neural network; computer vision; scale invariant feature transform; histogram of oriented gradients
Subject
Computer Science and Mathematics, Computer Science
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
