Face deidentification with generative deep models (FaceGEN)

About

The research project “Face deidentification with Generative Deep Models” (FaceGEN), strives to conduct research on deidentification technology and visual privacy with a particular focus on deep learning, which has recently been shown to be a highly effective tool for various computer vision and machine learning problems. Our goal is to develop deep generative models and conditional face synthesis techniques that can be used for deidentification and privacy protection with still images, but also with video, where multiple faces in cluttered and unconstrained scenes may appear in the data. The main tangible results of the project will be novel generative deep models and input-conditioned image synthesis techniques that are able to deidentify all parts of the facial data photo-realistically and ensure higher levels of privacy for users.

FaceGEN is a fundematal research project funded by the Slovenian Research Agency (ARRS) in in the period: 1.7.2019 – 31.6.2022 (1,66 FTE per year).

The Principal Investigator (PI) of FaceGEN is Assoc. Prof. Vitomir Štruc, PhD.

Link to SICRIS: Follow me.


Project overview

FaceGEN is structured into 7 work packages:

  • WP1: Project management
  • WP2: Prerequisites
  • WP3: Face synthesis with generative models
  • WP4: Face alteration with gradient ascent
  • WP5: Face deidentification in video data
  • WP6: Demo room and exploitation
  • WP7: Dissemination

The R&D work on this work packages is expected to result in:

  • Formal privacy schemes based on deep models
  • Photo-realistic face synthesis techniques
  • Novel deidentification methods for image and video data

Project phases

  • Year 1: Activities on work packages WP1, WP2, WP3, WP4, WP7
  • Year 2: Activities on work packages WP1, WP3, WP4, WP7
  • Year 3: Activities on work packages WP1, WP5, WP6, WP7

Partners

FaceGEN is conducted jointly by:


Participating researchers

International Advisory Committee


Project publications

  • R. Tolosana, C. Rathgeb, R. Vera-Rodriguez, C. Busch, L. Verdilova, S. Lyu, H.H. Nguyen, J. Yamagishi, I. Echizen, P. Rot, K. Grm, V. Štruc, A. Dantcheva, Z. Akhtar, S. Romero-Tapiador, J. Fierrez, A. Morales, J. Ortega-Garcia, E. Kindt, C. Jasserand, T. Kalvet, M. Tiits, Future Trends in Digital Face Manipulation and Detection, in: C. Rathgeb, R. Tulsana, R. Vera-Rodriguez, C. Busch (Eds.), Hanbook of Digital Face Manipulation and Detection, Springer, pp. 464-482, 2022 [PDF]
  • Peter Rot, Peter Peer, Vitomir Štruc, Detecting Soft-Biometric Privacy Enhancement, in: C. Rathgeb, R. Tulsana, R. Vera-Rodriguez, C. Busch (Eds.), Hanbook of Digital Face Manipulation and Detection, Springer, pp. 391-412, 2022 [PDF]
  • Dailé Osorio-Roig, Christian Rathgeb, Pawel Drozdowski, Philipp Terhörst, Vitomir Štruc, Christoph Busch, An Attack on Feature Level-based Facial Soft-biometric Privacy Enhancement, arXiv preprint, 2021 [PDF]
  • Blaž Meden, Peter Rot, Philipp Terhorst, Naser Damer, Arjan Kuijper, Arun Ross, Walter Scheirer, Peter Peer, Vitomir Štruc, Privacy-Enhancing Face Biometrics: A Comprehensive Survey, IEEE Transactions on Information Forensics and Security (SCI IF 2020: 7.178), 2021 [PDF]
  • Blaz Bortolato, Marija Ivanovska, Peter Rot, Janez Krizaj, Philipp Terhörst, Naser Damer, Peter Peer, Vitomir Štruc, Learning privacy-enhancing face representations through feature disentanglement, In: Proceedings of FG 2020, Buenos Aires, Argentina, 2020 [PDF].
  • Philipp Terhörst, Kevin Riehl, Naser Damer, Peter Rot, Blaž Bortolato, Florian Kirchbuchner, Vitomir Štruc, Arjan Kuijper, PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information UnitsIEEE Access (SCI IF 2019: 3.745), 2020 [PDF].
  • Philipp Terhörst, Marco Huber, Naser Damer, Peter Rot, Florian Kirchbuchner, Vitomir Štruc, Arjan Kuijper, Privacy Evaluation Protocols for the Evaluation of Soft-Biometric Privacy-Enhancing Technologies, In: Proceedings of BIOSIG 2020, Darmstadt, Germany, 2020 [PDF].

Patent

ŠTRUC, Vitomir, IVANOVSKA, Marija, ROT, Peter, PEER, Peter, ČERNE, Tomaž, ŽGANEC GROS, Jerneja. Postopek za prepoznavanje identitete na podlagi obraznih lastnosti z upoštevanjem varovanja zasebnosti : patent SI 25987 A, 2021-10-29. Ljubljana: Urad RS za intelektualno lastnino, 2021.


Master Theses

Žiga Babnik, Face Deidentification using Face Swapping, Master Thesis, Co-supervisors: Peter Peer and Vitomir Štruc, 2021 [PDF]

Nejc Sušin, Improving a deidentification model using generative adversarial networks, Master Thesis, Co-supervisors: Peter Peer and Vitomir Štruc, 2019 [PDF]


Ivited Talks

  • Peter Peer, Overview of privacy-enhancing face biometrics, Keynote Talk, ELMAR2021: 63rd International Symposium ELMAR-2021, Zadar, Croatia, 13-15 September 2021. Zadar: Croatian Society Electronics in Marine, 2021.
  • Peter Peer, Generative deep neural networks for face deidentification, Keynote Talk, IC3A 2020.
  • Peter Peer, From sclera recognition to face biometrics privacy-enhancing techniques and soft-biometric modalities, lecture at Silesian University of Technology, Gliwice, Poland, 13. 10. 2021.
  • Peter Peer, Research conducted in the Computer Vision Laboratory with focus on biometrics, lecture at Galgotias University, New Delhi, India, 3rd February 2020.

Deliverables

Some of project deliverables represent reports, which (in the first stage) are made publicly available to the funding agency only. The deliverables will be used as the basis for the project publications and will be released to the general public after publication in a peer reviewed venue. The deliverables can be accessed from the FaceGEN Deliverable page.


Funding agency