DeepFake detection using anomaly detection methods (DeepFake DAD)
To prevent illicit activities enabled by deepfake technologies, it is paramount to have highly automated and reliable means of detecting deepfakes at one’s disposal. Such detection technology not only enables efficient (large-scale) screening of image and video content but also allows non-experts to identify whether a given video or image is real or manipulated. Within the research project Deepfake detection using anomaly detection methods (DeepFake DAD) we aim to address this issue and conduct research on fundamentally novel methods for deepfake detection that address the deficiencies of current solutions in this problem domain. Existing deepfake detectors rely on (semi-)handcrafted features that have been shown to work against a predefined set of publicly available/known deepfake generation methods. However, detection techniques developed in this manner are vulnerable (i.e., unable to detect) to unknown or unseen (future) deepfake generation methods. The goal of DeepFake DAD is, therefore, to develop detection models that can be trained in a semi-supervised or unsupervised manner without relying on training samples from publicly available deepfake generation techniques, i.e., within so-called anomaly detection frameworks trainable in a one-class learning regime. The expected main tangible result of the research project are highly robust deepfake detection solutions that outperform the current state-of-the-art in terms of generalization capabilities and can assist end-users and platform providers to automatically detect tampered imagery and video, allowing them to act accordingly and avoid the negative personal, societal, economic, and political implications of widespread, undetectable fake footage.
Duration: 2023 – 2026, Project leader (PI): Vitomir Štruc
Project type: ARIS fundamental research project
Project website: DeepFake DAD homepage
Mechanistic Interpretability for eXplainable Biometric Artificial Intelligence (MIXBAI)
The Mechanistic Interpretability for eXplainable Biometric Artificial Intelligence (MIXBAI) project aims to perform research in the fields of AI explainability and interpretability, and use it specifically on applications of AI technologies in the field of biometrics. Our goal is to develop novel interpretability methods to interpret the inner workings of biometric AI systems, explain their decisions, and enable their use in a fair, unbiased, and trustworthy manner.
Duration: 2023 – 2026, Project leader (PI): Klemen Grm
Project type: ARRS fundamental research project
Project website: MIXBAI homepage
Deep generative models for beauty and fashion (DeepBeauty)
The Deep generative models for beauty and fashion (DeepBeauty) project aims to contribute to research on image generation and editing technology with a particular focus on deep learning methodologies, which have recently been shown to be a highly convenient and effective tool for this task. The goal of DeepBeauty is to develop novel (flexible and robust) mechanisms for image editing tailored towards the needs of the beauty and fashion industries, capable of altering certain parts of the input images in accordance with predefined target appearances (e.g., an example of specific makeup, an image of a model wearing a fashion item, clothing or accessory). The main planned tangible result of the project is novel and highly robust virtual try-on technology based on original approaches to face and body editing.
Duration: 2020 – 2024
Project leader (PI): Vitomir Štruc
Project type: ARRS fundamental research project
Project website: DeepBeauty homepage
Development of Slovene in a Digital Environment (DSDE)
With the project titled Development of Slovene in a Digital Environment (DSDE), which is financed by the Slovenian Ministry of Culture, and the European Regional Development Fund, Slovenia has recognized the importance of developing modern language technologies for the Slovene language. The project strives to meet the needs for computational tools and services in the field of language technologies for Slovene, which will be used by research organizations, companies, and the general public. The development of language technologies is crucial for language survival in the digital age; these tools are the only way to keep participating in the new ways of communication, work and leisure that will be available in the future. The project Development of Slovene in a Digital Environment is co-financed by the Republic of Slovenia and the European Union from the European Regional Development Fund.
Duration: 2020 – 2023, Project leader at FEE: Simon Dobrišek
Project type: ERDF Project
National research Program: Metrology and Biometric Systems, P2-0250 (B)
Within the scope of the research programme researchers from two universities and R&D oriented SME conduct fundamental as well as applied research in selected areas related to measurement on the one hand and to the analysis and modelling of measurement data on the other hand. The research programme puts a particular focus on magnetic measurements, environmental measurements, biometric measurements and speech modelling.
Duration: 2018-2023, Programme leader (PI): Vitomir Štruc
Project type: ARRS research programme
Face deidentification with generative deep models (FaceGEN)
The research project “Face deidentification with Generative Deep Models” (FaceGEN), strives to conduct research on deidentification technology with a particular focus on deep learning, which has recently been shown to be a highly effective tool for various computer vision and machine learning problems. Our goal is to develop deep generative models and conditional face synthesis techniques that can be used for deidentification with still images, but also with video, where multiple faces in cluttered and unconstrained scenes may appear in the data. The main tangible results of the project will be novel generative deep models and input-conditioned image synthesis techniques that are able to deidentify all parts of the facial data photo-realistically.
Duration: 2019 – 2022, Project leader (PI): Vitomir Štruc
Project type: ARRS fundamental research project
Project website: FaceGEN homepage
A neural network solution to segmentation and recovery of superquadric models from 3D image data (SQ CNN)
The objective of this research proposal is to develop a CNN solution for real-time segmentation and superquadric model recovery from large 3D point clouds. In addition to the development of CNNs for segmentation and model recovery of superquadrics from 3D point clouds, we would like to find out if these CNNs for segmentation and model recovery from 3D point clouds could be adapted to reconstruction from 2D intensity images.
There is ample evidence by current research that the marriage of 3D data and models with CNN computational paradigm is adequate but only starting. Our motivation, however, is to develop a general purpose CNN based solution which can give for a given selected scene, defined with corresponding 3D point clouds and/or intensity images, its description in terms of supequadrics as part-level models. The output of our proposed solution would therefore be the parameter values of an unspecified number of superquadrics, which are necessary to describe a given scene. To our knowledge, no method exists yet for recovery of part-level volumetric models, such as superquadrics from 3D point clouds using CNNs. This research project would contribute to the growing field of 3D recovery and modeling using CNNs.
Duration: 2018 – 2021, Project leader at LMI: Vitomir Štruc
Project type: ARRS fundamental research project
Project webpage: SQ CNN homepage
Detection of inconsistencies in complex visual data using deep learning (DIVID)
The project addresses research issues important to several research areas. Our primary goal is to go beyond the supervised deep learning for the specific case of automated detection of anomalies in visual data. We therefore expect to go beyond the state-of-the-art in this specific research area. However, to achieve this goal, we will develop several concepts related to the fundamental understanding of deep learning, which could be applicable in deep learning approaches in general, thus applicable on various computer vision problems.
The project will primarily focus on the development of basic methods for semi- and unsupervised learning. We will address adversarial learning of generative deep networks and show novel methods that incorporate compositional properties in deep models. Using the properties of compositional hierarchies, we will improve the understanding of deep networks. Although the computer vision and deep learning community has started investigating approaches that do not require a huge number of labelled training data, the research field still predominantly relies on supervised learning, so we expect that the results of the proposed project will have a significant impact to the future development in this research area.
Duration: 2018 – 2021, Project leader at LMI: Janez Perš
Project type: ARRS fundamental research project
Project homepage: DIVID homepage
OptiLEX: Research and optimization of lexical resources for embedded speech technology implementations (OptiLEX)
In the OptiLEX applied project, the focus will be on pronunciation dictionaries and on research and implementation of a module that will enable effective presentation of lexical resources and be able to function in speech engines on embedded platforms. A number of problems will be addressed: the need to operate in real time; the need to store language resources in a compact manner; and finally, the need for a small language-resource footprint in random-access memory.
The objective of the OptiLEX project is to study procedures for optimizing the computer presentation of lexical resources for inflection-rich languages that may be used for speech recognition and synthesis on embedded platforms. The procedures will be implemented and validated on the hardware of an embedded mobile terminal.
The project group will seek effective procedures for reducing redundancy in the presentation and computerized storage of lexical resources for inflection-rich language groups, which will enable fast and high-quality conversion of grapheme-based entry of words into a phonetic transcription and vice versa by utilizing as little memory as possible.
Duration: 2018 – 2021, Project leader at LMI: Simon Dobrišek
Project type: ARRS fundamental research project
Super-resolution for face recognition (SuperFACE)
In the scope of this bilateral project with the University of Notre Dame we research and develop novel models for face super-resolution.
Duration: 2018 – 2019, Project leader at LMI: Vitomir Štruc
Project type: ARRS bilateral project
Past Projects (recent, selected)
RESPECT: Rules, Expectations & Security through Privacy-Enhanced Convenient Technologies
RESPECT seeks to investigate if the current and foreseeable implementation of ICTs in surveillance is indeed “in balance” and, where a lack of balance may exist or is perceived by citizens not to exist, the project explores options for redressing the balance through a combination of Privacy-Enhancing Technologies and operational approaches. Investigating at least five key sectors not yet tackled by other recent projects researching surveillance (CCTV, database mining and interconnection, on-line social network analysis, RFID& geo-location/sensor devices, financial tracking), RESPECT will also carry out quantitative and qualitative research on citizens’ awareness and attitudes to surveillance. RESPECT will produce tools that would enable policy makers to understand the socio-cultural as well as the operational and economic impact of surveillance systems. The project will also produce operational guidelines incorporating privacy by design approaches which would enable law enforcement agencies to deploy surveillance systems with lowest privacy risk possible and maximum security gain to citizens.
Duration: 2012-2015, PI at LMI: Simon Dobrišek
Project type: EU FP 7 CP
SMART: Scalable Measures for Automated Recognition Technologies
The SMART project addresses the questions of automated decision taking with respect to the “smart surveillance” technologies in a society where privacy and data protection are fundamental rights. The risks and opportunities inherent to the use of smart surveillance will be evaluated and a number of technical, procedural and legal options for safeguards will be developed. SMART aims to create a toolkit which would inform system designers, policy makers and legislative bodies across Europe and beyond.
Duration: 2011-2014, PI at LMI: Simon Dobrišek
Project type: EU FP 7 CP
EU Social fund and MIZS post-doctoral Project: 3D Face Recognition in Real World Settings (3D-For-Real)
(Abstract available ony in slovenian) Kljub številnim raziskavam in dosegljivi literaturi na temo razpoznavanja 3D slik obrazov, ostaja razvoj in implementacija robustnega sistema za razpoznavanje obrazov v nenadzorovanih razmerah zajema 3D slik specifičen problem, ki zahteva rešitev vrste težav. Cilj raziskovalnega projekta je zato predlagati sistem za razpoznavanje 3D slik obrazov, ki bo omogočal zanesljivo razpoznavanje obrazov tudi v najzahtevnejših okoljih, to je v prisotnosti večjega števila motečih dejavnikov, ki so lahko navzoči pri zajemu 3D slik in vplivajo na izgled obrazov na 3D slikah. Osredotočili se bomo na razvoj postopkov za učinkovito reševanje problemov, ki nastanejo ob prisotnosti motečih dejavnikov. Posebno pozornost bomo namenili reševanju problemov robustnega razpoznavanja obrazov pri interakciji dejavnikov kot so zorni kot oz. orientacija obrazov, delna prekrivanja obraznega področja, izrazne spremembe in oddaljenost obrazov od senzorja.
Duration: 2014-2015, Project leader: Janez Križaj
Project type: Postdoctoral project