2023 |
Eyiokur, Fevziye Irem; Kantarci, Alperen; Erakin, Mustafa Ekrem; Damer, Naser; Ofli, Ferda; Imran, Muhammad; Križaj, Janez; Salah, Albert Ali; Waibel, Alexander; Štruc, Vitomir; Ekenel, Hazim K. A Survey on Computer Vision based Human Analysis in the COVID-19 Era Članek v strokovni reviji V: Image and Vision Computing, vol. 130, no. 104610, str. 1-19, 2023. Povzetek | Povezava | BibTeX | Oznake: COVID-19, face, face alignment, face analysis, face image processing, face image quality assessment, face landmarking, face recognition, face verification, human analysis, masked face analysis @article{IVC2023, The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of (i) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and (ii) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given at the end of the survey. This work is intended to have a broad appeal and be useful not only for computer vision researchers but also the general public. |
2019 |
Krizaj, Janez; Peer, Peter; Struc, Vitomir; Dobrisek, Simon Simultaneous multi-decent regression and feature learning for landmarking in depth image Članek v strokovni reviji V: Neural Computing and Applications, 2019, ISBN: 0941-0643. Povzetek | Povezava | BibTeX | Oznake: 3d, biometrics, depth data, face alignment, face analysis, landmarking @article{Krizaj3Docalization, Face alignment (or facial landmarking) is an important task in many face-related applications, ranging from registration, tracking, and animation to higher-level classification problems such as face, expression, or attribute recognition. While several solutions have been presented in the literature for this task so far, reliably locating salient facial features across a wide range of posses still remains challenging. To address this issue, we propose in this paper a novel method for automatic facial landmark localization in 3D face data designed specifically to address appearance variability caused by significant pose variations. Our method builds on recent cascaded regression-based methods to facial landmarking and uses a gating mechanism to incorporate multiple linear cascaded regression models each trained for a limited range of poses into a single powerful landmarking model capable of processing arbitrary-posed input data. We develop two distinct approaches around the proposed gating mechanism: (1) the first uses a gated multiple ridge descent mechanism in conjunction with established (hand-crafted) histogram of gradients features for face alignment and achieves state-of-the-art landmarking performance across a wide range of facial poses and (2) the second simultaneously learns multiple-descent directions as well as binary features that are optimal for the alignment tasks and in addition to competitive landmarking results also ensures extremely rapid processing. We evaluate both approaches in rigorous experiments on several popular datasets of 3D face images, i.e., the FRGCv2 and Bosphorus 3D face datasets and image collections F and G from the University of Notre Dame. The results of our evaluation show that both approaches compare favorably to the state-of-the-art, while exhibiting considerable robustness to pose variations. |
2018 |
Križaj, Janez; Emeršič, Žiga; Dobrišek, Simon; Peer, Peter; Štruc, Vitomir Localization of Facial Landmarks in Depth Images Using Gated Multiple Ridge Descent Proceedings Article V: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), str. 1–8, IEEE 2018. Povzetek | Povezava | BibTeX | Oznake: 3d face, 3d landmarking, face alignment, face landmarking, gated ridge descent @inproceedings{krivzaj2018localization, A novel method for automatic facial landmark localization is presented. The method builds on the supervised descent framework, which was shown to successfully localize landmarks in the presence of large expression variations and mild occlusions, but struggles when localizing landmarks on faces with large pose variations. We propose an extension of the supervised descent framework that trains multiple descent maps and results in increased robustness to pose variations. The performance of the proposed method is demonstrated on the Bosphorus, the FRGC and the UND data sets for the problem of facial landmark localization from 3D data. Our experimental results show that the proposed method exhibits increased robustness to pose variations, while retaining high performance in the case of expression and occlusion variations. |
Banerjee, Sandipan; Brogan, Joel; Krizaj, Janez; Bharati, Aparna; RichardWebster, Brandon; Struc, Vitomir; Flynn, Patrick J.; Scheirer, Walter J. To frontalize or not to frontalize: Do we really need elaborate pre-processing to improve face recognition? Proceedings Article V: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), str. 20–29, IEEE 2018. Povzetek | Povezava | BibTeX | Oznake: face alignment, face recognition, landmarking @inproceedings{banerjee2018frontalize, Face recognition performance has improved remarkably in the last decade. Much of this success can be attributed to the development of deep learning techniques such as convolutional neural networks (CNNs). While CNNs have pushed the state-of-the-art forward, their training process requires a large amount of clean and correctly labelled training data. If a CNN is intended to tolerate facial pose, then we face an important question: should this training data be diverse in its pose distribution, or should face images be normalized to a single pose in a pre-processing step? To address this question, we evaluate a number of facial landmarking algorithms and a popular frontalization method to understand their effect on facial recognition performance. Additionally, we introduce a new, automatic, single-image frontalization scheme that exceeds the performance of the reference frontalization algorithm for video-to-video face matching on the Point and Shoot Challenge (PaSC) dataset. Additionally, we investigate failure modes of each frontalization method on different facial yaw using the CMU Multi-PIE dataset. We assert that the subsequent recognition and verification performance serves to quantify the effectiveness of each pose correction scheme. |
2016 |
Križaj, Janez; Dobrišek, Simon; Mihelič, France; Štruc, Vitomir Facial Landmark Localization from 3D Images Proceedings Article V: Proceedings of the Electrotechnical and Computer Science Conference (ERK), Portorož, Slovenia, 2016. Povzetek | BibTeX | Oznake: 3D face data, 3d landmarking, Bosphorus, face alignment, face image processing, facial landmarking, SDM, supervised descent framework @inproceedings{ERK2016Janez, A novel method for automatic facial landmark localization is presented. The method builds on the supervised descent framework, which was shown to successfully localize landmarks in the presence of large expression variations and mild occlusions, but struggles when localizing landmarks on faces with large pose variations. We propose an extension of the supervised descent framework which trains multiple descent maps and results in increased robustness to pose variations. The performance of the proposed method is demonstrated on the Bosphorus database for the problem of facial landmark localization from 3D data. Our experimental results show that the proposed method exhibits increased robustness to pose variations, while retaining high performance in the case of expression and occlusion variations. |
Fabijan, Sebastjan; Štruc, Vitomir Vpliv registracije obraznih področij na učinkovitost samodejnega razpoznavanja obrazov: študija z OpenBR Proceedings Article V: Proceedings of the Electrotechnical and Computer Science Conference (ERK), 2016. Povzetek | Povezava | BibTeX | Oznake: 4SF, biometrics, face alignment, face recognition, LFW, OpenBR, performance evaluation @inproceedings{ERK2016_Seba, Razpoznavanje obrazov je v zadnjih letih postalo eno najuspešnejših področij samodejne, računalniško podprte analize slik, ki se lahko pohvali z različnimi primeri upor-abe v praksi. Enega ključnih korakav za uspešno razpoznavanje predstavlja poravnava obrazov na slikah. S poravnavo poskušamo zagotoviti neodvisnost razpozn-av-an-ja od sprememb zornih kotov pri zajemu slike, ki v slikovne podatke vnašajo visoko stopnjo variabilnosti. V prispevku predstavimo tri postopke poravnavanja obrazov (iz literature) in proučimo njihov vpliv na uspešnost razpoznavanja s postopki, udejanjenimi v odprtokodnem programskem ogrodju Open Source Biometric Recognition (OpenBR). Vse poizkuse izvedemo na podatkovni zbirki Labeled Faces in the Wild (LFW). |
Objave
2023 |
A Survey on Computer Vision based Human Analysis in the COVID-19 Era Članek v strokovni reviji V: Image and Vision Computing, vol. 130, no. 104610, str. 1-19, 2023. |
2019 |
Simultaneous multi-decent regression and feature learning for landmarking in depth image Članek v strokovni reviji V: Neural Computing and Applications, 2019, ISBN: 0941-0643. |
2018 |
Localization of Facial Landmarks in Depth Images Using Gated Multiple Ridge Descent Proceedings Article V: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), str. 1–8, IEEE 2018. |
To frontalize or not to frontalize: Do we really need elaborate pre-processing to improve face recognition? Proceedings Article V: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), str. 20–29, IEEE 2018. |
2016 |
Facial Landmark Localization from 3D Images Proceedings Article V: Proceedings of the Electrotechnical and Computer Science Conference (ERK), Portorož, Slovenia, 2016. |
Vpliv registracije obraznih področij na učinkovitost samodejnega razpoznavanja obrazov: študija z OpenBR Proceedings Article V: Proceedings of the Electrotechnical and Computer Science Conference (ERK), 2016. |