Conditional face image manipulation detection: combining algorithm and human examiner decisions

Mathias Ibsen, Robert Nichols, Christian Rathgeb, David J. Robertson, Josh P. Davis, Frøy Løvåsdal, Kiran Raja, Ryan E. Jenkins, Christoph Busch

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

12 Downloads (Pure)

Abstract

It has been shown that digitally manipulated face images can pose a security threat to automated authentication systems (e.g., when such systems are used for border control). In such scenarios, a malicious actor can, in many countries, apply for an identity document using a manipulated face image, which can then be used to gain fraudulent access to a system. Research has shown that humans and algorithms struggle to detect digitally manipulated face images, especially when the type of manipulation is unknown or when evaluated across multiple types of manipulations. In this work, we consider the detection performance of algorithms and humans on datasets consisting of retouched, face swapped and morphed images. Specifically, we investigate the joint performance of algorithms and humans in a differential detection scenario where both a trusted and suspected image are presented simultaneously. To this end, we propose a conditional face image manipulation detection approach where the human decision is only considered when the algorithm is unsure about the decision outcome. The results show that the automated algorithm performs better than the human detectors and that combining the decisions of algorithms and humans, in general, leads to an increased detection performance. To our knowledge, this is the first study to explore the joint detection performance of algorithms and humans in a differential face manipulation detection scenario and when using a variety of face image manipulations.
Original languageEnglish
Title of host publicationProceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security
Place of PublicationNew York, NY
Pages41-46
Number of pages6
ISBN (Electronic)9798400706370
DOIs
Publication statusPublished - 24 Jun 2024
Event12th ACM Workshop on Information Hiding and Multimedia Securit - Spain, Baiona, Spain
Duration: 24 Jun 202426 Jun 2024
https://www.ihmmsec.org/cms/home/home173.html?acceptCookie=1&idart=205&idcat=69&changelang=2

Conference

Conference12th ACM Workshop on Information Hiding and Multimedia Securit
Abbreviated titleIH&MMSec 2024
Country/TerritorySpain
CityBaiona
Period24/06/2426/06/24
Internet address

Keywords

  • algorithm performance
  • digital forensics
  • face manipulation detection
  • human performance
  • information fusion

Fingerprint

Dive into the research topics of 'Conditional face image manipulation detection: combining algorithm and human examiner decisions'. Together they form a unique fingerprint.

Cite this