2023 |
Vitek, Matej; Das, Abhijit; Lucio, Diego Rafael; Jr., Luiz Antonio Zanlorensi; Menotti, David; Khiarak, Jalil Nourmohammadi; Shahpar, Mohsen Akbari; Asgari-Chenaghlu, Meysam; Jaryani, Farhang; Tapia, Juan E.; Valenzuela, Andres; Wang, Caiyong; Wang, Yunlong; He, Zhaofeng; Sun, Zhenan; Boutros, Fadi; Damer, Naser; Grebe, Jonas Henry; Kuijper, Arjan; Raja, Kiran; Gupta, Gourav; Zampoukis, Georgios; Tsochatzidis, Lazaros; Pratikakis, Ioannis; Kumar, S. V. Aruna; Harish, B. S.; Pal, Umapada; Peer, Peter; Štruc, Vitomir Exploring Bias in Sclera Segmentation Models: A Group Evaluation Approach Članek v strokovni reviji V: IEEE Transactions on Information Forensics and Security, vol. 18, str. 190-205, 2023, ISSN: 1556-6013. Povzetek | Povezava | BibTeX | Oznake: bias, biometrics, fairness, group evaluation, ocular, sclera, sclera segmentation, segmentation @article{TIFS_Sclera2022, Bias and fairness of biometric algorithms have been key topics of research in recent years, mainly due to the societal, legal and ethical implications of potentially unfair decisions made by automated decision-making models. A considerable amount of work has been done on this topic across different biometric modalities, aiming at better understanding the main sources of algorithmic bias or devising mitigation measures. In this work, we contribute to these efforts and present the first study investigating bias and fairness of sclera segmentation models. Although sclera segmentation techniques represent a key component of sclera-based biometric systems with a considerable impact on the overall recognition performance, the presence of different types of biases in sclera segmentation methods is still underexplored. To address this limitation, we describe the results of a group evaluation effort (involving seven research groups), organized to explore the performance of recent sclera segmentation models within a common experimental framework and study performance differences (and bias), originating from various demographic as well as environmental factors. Using five diverse datasets, we analyze seven independently developed sclera segmentation models in different experimental configurations. The results of our experiments suggest that there are significant differences in the overall segmentation performance across the seven models and that among the considered factors, ethnicity appears to be the biggest cause of bias. Additionally, we observe that training with representative and balanced data does not necessarily lead to less biased results. Finally, we find that in general there appears to be a negative correlation between the amount of bias observed (due to eye color, ethnicity and acquisition device) and the overall segmentation performance, suggesting that advances in the field of semantic segmentation may also help with mitigating bias. |
2016 |
Scheirer, Walter; Flynn, Patrick; Ding, Changxing; Guo, Guodong; Štruc, Vitomir; Jazaery, Mohamad Al; Dobrišek, Simon; Grm, Klemen; Tao, Dacheng; Zhu, Yu; Brogan, Joel; Banerjee, Sandipan; Bharati, Aparna; Webster, Brandon Richard Report on the BTAS 2016 Video Person Recognition Evaluation Proceedings Article V: Proceedings of the IEEE International Conference on Biometrics: Theory, Applications ans Systems (BTAS), IEEE, 2016. Povzetek | BibTeX | Oznake: biometrics, competition, face recognition, group evaluation, PaSC, performance evaluation @inproceedings{BTAS2016, This report presents results from the Video Person Recognition Evaluation held in conjunction with the 8th IEEE International Conference on Biometrics: Theory, Applications, and Systems (BTAS). Two experiments required algorithms to recognize people in videos from the Pointand- Shoot Face Recognition Challenge Problem (PaSC). The first consisted of videos from a tripod mounted high quality video camera. The second contained videos acquired from 5 different handheld video cameras. There were 1,401 videos in each experiment of 265 subjects. The subjects, the scenes, and the actions carried out by the people are the same in both experiments. An additional experiment required algorithms to recognize people in videos from the Video Database of Moving Faces and People (VDMFP). There were 958 videos in this experiment of 297 subjects. Four groups from around the world participated in the evaluation. The top verification rate for PaSC from this evaluation is 0:98 at a false accept rate of 0:01 — a remarkable advancement in performance from the competition held at FG 2015. |
2015 |
Beveridge, Ross; Zhang, Hao; Draper, Bruce A; Flynn, Patrick J; Feng, Zhenhua; Huber, Patrik; Kittler, Josef; Huang, Zhiwu; Li, Shaoxin; Li, Yan; Štruc, Vitomir; Križaj, Janez; others, Report on the FG 2015 video person recognition evaluation Proceedings Article V: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (IEEE FG), str. 1–8, IEEE 2015. Povzetek | Povezava | BibTeX | Oznake: biometrics, competition, face verification, FG, group evaluation, PaSC, performance evaluation @inproceedings{beveridge2015report, This report presents results from the Video Person Recognition Evaluation held in conjunction with the 11th IEEE International Conference on Automatic Face and Gesture Recognition. Two experiments required algorithms to recognize people in videos from the Point-and-Shoot Face Recognition Challenge Problem (PaSC). The first consisted of videos from a tripod mounted high quality video camera. The second contained videos acquired from 5 different handheld video cameras. There were 1401 videos in each experiment of 265 subjects. The subjects, the scenes, and the actions carried out by the people are the same in both experiments. Five groups from around the world participated in the evaluation. The video handheld experiment was included in the International Joint Conference on Biometrics (IJCB) 2014 Handheld Video Face and Person Recognition Competition. The top verification rate from this evaluation is double that of the top performer in the IJCB competition. Analysis shows that the factor most effecting algorithm performance is the combination of location and action: where the video was acquired and what the person was doing. |
2014 |
Beveridge, Ross; Zhang, Hao; Flynn, Patrick; Lee, Yooyoung; Liong, Venice Erin; Lu, Jiwen; de Angeloni, Marcus Assis; de Pereira, Tiago Freitas; Li, Haoxiang; Hua, Gang; Štruc, Vitomir; Križaj, Janez; Phillips, Jonathon The ijcb 2014 pasc video face and person recognition competition Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB), str. 1–8, IEEE 2014. Povzetek | Povezava | BibTeX | Oznake: biometrics, competition, face recognition, group evaluation, IJCB, PaSC, performance evaluation @inproceedings{beveridge2014ijcb, The Point-and-Shoot Face Recognition Challenge (PaSC) is a performance evaluation challenge including 1401 videos of 265 people acquired with handheld cameras and depicting people engaged in activities with non-frontal head pose. This report summarizes the results from a competition using this challenge problem. In the Video-to-video Experiment a person in a query video is recognized by comparing the query video to a set of target videos. Both target and query videos are drawn from the same pool of 1401 videos. In the Still-to-video Experiment the person in a query video is to be recognized by comparing the query video to a larger target set consisting of still images. Algorithm performance is characterized by verification rate at a false accept rate of 0:01 and associated receiver operating characteristic (ROC) curves. Participants were provided eye coordinates for video frames. Results were submitted by 4 institutions: (i) Advanced Digital Science Center, Singapore; (ii) CPqD, Brasil; (iii) Stevens Institute of Technology, USA; and (iv) University of Ljubljana, Slovenia. Most competitors demonstrated video face recognition performance superior to the baseline provided with PaSC. The results represent the best performance to date on the handheld video portion of the PaSC. |
2013 |
Štruc, Vitomir; Gros, Jeneja Žganec; Dobrišek, Simon; Pavešić, Nikola Exploiting representation plurality for robust and efficient face recognition Proceedings Article V: Proceedings of the 22nd Intenational Electrotechnical and Computer Science Conference (ERK'13), str. 121–124, Portorož, Slovenia, 2013. Povzetek | Povezava | BibTeX | Oznake: competition, erk, face recognition, face verification, group evaluation, ICB, mobile biometrics, MOBIO, performance evaluation @inproceedings{ERK2013_Struc, The paper introduces a novel approach to face recognition that exploits plurality of representation to achieve robust face recognition. The proposed approach was submitted as a representative of the University of Ljubljana and Alpineon d.o.o. to the 2013 face recognition competition that was held in conjunction with the IAPR International Conference on Biometrics and achieved the best overall recognition results among all competition participants. Here, we describe the basic characteristics of the submitted approach, elaborate on the results of the competition and, most importantly, present some general findings made during our development work that are of relevance to the broader (face recognition) research community. |
Günther, Manuel; Costa-Pazo, Artur; Ding, Changxing; Boutellaa, Elhocine; Chiachia, Giovani; Zhang, Honglei; de Angeloni, Marcus Assis; Štruc, Vitomir; Khoury, Elie; Vazquez-Fernandez, Esteban; others, The 2013 face recognition evaluation in mobile environment Proceedings Article V: Proceedings of the IAPR International Conference on Biometrics (ICB), str. 1–7, IAPR 2013. Povzetek | Povezava | BibTeX | Oznake: biometrics, competition, face recognition, face verification, group evaluation, mobile biometrics, MOBIO, performance evaluation @inproceedings{gunther20132013, Automatic face recognition in unconstrained environments is a challenging task. To test current trends in face recognition algorithms, we organized an evaluation on face recognition in mobile environment. This paper presents the results of 8 different participants using two verification metrics. Most submitted algorithms rely on one or more of three types of features: local binary patterns, Gabor wavelet responses including Gabor phases, and color information. The best results are obtained from UNILJ-ALP, which fused several image representations and feature types, and UCHU, which learns optimal features with a convolutional neural network. Additionally, we assess the usability of the algorithms in mobile devices with limited resources. |
2010 |
Poh, Norman; Chan, Chi Ho; Kittler, Josef; Marcel, Sebastien; Cool, Christopher Mc; Rua, Enrique Argones; Castro, Jose Luis Alba; Villegas, Mauricio; Paredes, Roberto; Struc, Vitomir; others, An evaluation of video-to-video face verification Članek v strokovni reviji V: IEEE Transactions on Information Forensics and Security, vol. 5, no. 4, str. 781–801, 2010. Povzetek | Povezava | BibTeX | Oznake: biometrics, competition, face recognition, face verification, group evaluation, video @article{poh2010evaluation, Person recognition using facial features, e.g., mug-shot images, has long been used in identity documents. However, due to the widespread use of web-cams and mobile devices embedded with a camera, it is now possible to realize facial video recognition, rather than resorting to just still images. In fact, facial video recognition offers many advantages over still image recognition; these include the potential of boosting the system accuracy and deterring spoof attacks. This paper presents an evaluation of person identity verification using facial video data, organized in conjunction with the International Conference on Biometrics (ICB 2009). It involves 18 systems submitted by seven academic institutes. These systems provide for a diverse set of assumptions, including feature representation and preprocessing variations, allowing us to assess the effect of adverse conditions, usage of quality information, query selection, and template construction for video-to-video face authentication. |
Objave
2023 |
Exploring Bias in Sclera Segmentation Models: A Group Evaluation Approach Članek v strokovni reviji V: IEEE Transactions on Information Forensics and Security, vol. 18, str. 190-205, 2023, ISSN: 1556-6013. |
2016 |
Report on the BTAS 2016 Video Person Recognition Evaluation Proceedings Article V: Proceedings of the IEEE International Conference on Biometrics: Theory, Applications ans Systems (BTAS), IEEE, 2016. |
2015 |
Report on the FG 2015 video person recognition evaluation Proceedings Article V: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (IEEE FG), str. 1–8, IEEE 2015. |
2014 |
The ijcb 2014 pasc video face and person recognition competition Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB), str. 1–8, IEEE 2014. |
2013 |
Exploiting representation plurality for robust and efficient face recognition Proceedings Article V: Proceedings of the 22nd Intenational Electrotechnical and Computer Science Conference (ERK'13), str. 121–124, Portorož, Slovenia, 2013. |
The 2013 face recognition evaluation in mobile environment Proceedings Article V: Proceedings of the IAPR International Conference on Biometrics (ICB), str. 1–7, IAPR 2013. |
2010 |
An evaluation of video-to-video face verification Članek v strokovni reviji V: IEEE Transactions on Information Forensics and Security, vol. 5, no. 4, str. 781–801, 2010. |