2023 |
Emersic, Ziga; Ohki, Tetsushi; Akasaka, Muku; Arakawa, Takahiko; Maeda, Soshi; Okano, Masora; Sato, Yuya; George, Anjith; Marcel, Sébastien; Ganapathi, Iyyakutti Iyappan; Ali, Syed Sadaf; Javed, Sajid; Werghi, Naoufel; Işık, Selin Gök; Sarıtaş, Erdi; Ekenel, Hazim Kemal; Hudovernik, Valter; Kolf, Jan Niklas; Boutros, Fadi; Damer, Naser; Sharma, Geetanjali; Kamboj, Aman; Nigam, Aditya; Jain, Deepak Kumar; Cámara, Guillermo; Peer, Peter; Struc, Vitomir The Unconstrained Ear Recognition Challenge 2023: Maximizing Performance and Minimizing Bias Proceedings Article V: IEEE International Joint Conference on Biometrics (IJCB 2023), str. 1-10, Ljubljana, Slovenia, 2023. Povzetek | Povezava | BibTeX | Oznake: biometrics, competition, computer vision, deep learning, ear, ear biometrics, UERC 2023 @inproceedings{UERC2023, The paper provides a summary of the 2023 Unconstrained Ear Recognition Challenge (UERC), a benchmarking effort focused on ear recognition from images acquired in uncontrolled environments. The objective of the challenge was to evaluate the effectiveness of current ear recognition techniques on a challenging ear dataset while analyzing the techniques from two distinct aspects, i.e., verification performance and bias with respect to specific demographic factors, i.e., gender and ethnicity. Seven research groups participated in the challenge and submitted a seven distinct recognition approaches that ranged from descriptor-based methods and deep-learning models to ensemble techniques that relied on multiple data representations to maximize performance and minimize bias. A comprehensive investigation into the performance of the submitted models is presented, as well as an in-depth analysis of bias and associated performance differentials due to differences in gender and ethnicity. The results of the challenge suggest that a wide variety of models (e.g., transformers, convolutional neural networks, ensemble models) is capable of achieving competitive recognition results, but also that all of the models still exhibit considerable performance differentials with respect to both gender and ethnicity. To promote further development of unbiased and effective ear recognition models, the starter kit of UERC 2023 together with the baseline model, and training and test data is made available from: http://ears.fri.uni-lj.si/. |
Hrovatič, Anja; Peer, Peter; Štruc, Vitomir; Emeršič, Žiga Efficient ear alignment using a two-stack hourglass network Članek v strokovni reviji V: IET Biometrics , str. 1-14, 2023, ISSN: 2047-4938. Povzetek | Povezava | BibTeX | Oznake: biometrics, CNN, deep learning, ear, ear alignment, ear recognition @article{UhljiIETZiga, Ear images have been shown to be a reliable modality for biometric recognition with desirable characteristics, such as high universality, distinctiveness, measurability and permanence. While a considerable amount of research has been directed towards ear recognition techniques, the problem of ear alignment is still under-explored in the open literature. Nonetheless, accurate alignment of ear images, especially in unconstrained acquisition scenarios, where the ear appearance is expected to vary widely due to pose and view point variations, is critical for the performance of all downstream tasks, including ear recognition. Here, the authors address this problem and present a framework for ear alignment that relies on a two-step procedure: (i) automatic landmark detection and (ii) fiducial point alignment. For the first (landmark detection) step, the authors implement and train a Two-Stack Hourglass model (2-SHGNet) capable of accurately predicting 55 landmarks on diverse ear images captured in uncontrolled conditions. For the second (alignment) step, the authors use the Random Sample Consensus (RANSAC) algorithm to align the estimated landmark/fiducial points with a pre-defined ear shape (i.e. a collection of average ear landmark positions). The authors evaluate the proposed framework in comprehensive experiments on the AWEx and ITWE datasets and show that the 2-SHGNet model leads to more accurate landmark predictions than competing state-of-the-art models from the literature. Furthermore, the authors also demonstrate that the alignment step significantly improves recognition accuracy with ear images from unconstrained environments compared to unaligned imagery. |
2022 |
Dvoršak, Grega; Dwivedi, Ankita; Štruc, Vitomir; Peer, Peter; Emeršič, Žiga Kinship Verification from Ear Images: An Explorative Study with Deep Learning Models Proceedings Article V: International Workshop on Biometrics and Forensics (IWBF), str. 1–6, 2022. Povzetek | Povezava | BibTeX | Oznake: biometrics, CNN, deep learning, ear, ear biometrics, kinear, kinship, kinship recognition, transformer @inproceedings{KinEars, The analysis of kin relations from visual data represents a challenging research problem with important real-world applications. However, research in this area has mostly been limited to the analysis of facial images, despite the potential of other physical (human) characteristics for this task. In this paper, we therefore study the problem of kinship verification from ear images and investigate whether salient appearance characteristics, useful for this task, can be extracted from ear data. To facilitate the study, we introduce a novel dataset, called KinEar, that contains data from 19 families with each family member having from 15 to 31 ear images. Using the KinEar data, we conduct experiments using a Siamese training setup and 5 recent deep learning backbones. The results of our experiments suggests that ear images represent a viable alternative to other modalities for kinship verification, as 4 out of 5 considered models reach a performance of over 60% in terms of the Area Under the Receiver Operating Characteristics (ROC-AUC). |
2019 |
Emeršič, Žiga; V., A. Kumar S.; Harish, B. S.; Gutfeter, W.; Khiarak, J. N.; Pacut, A.; Hansley, E.; Segundo, M. Pamplona; Sarkar, S.; Park, H.; Nam, G. Pyo; Kim, I. J.; Sangodkar, S. G.; Kacar, U.; Kirci, M.; Yuan, L.; Yuan, J.; Zhao, H.; Lu, F.; Mao, J.; Zhang, X.; Yaman, D.; Eyiokur, F. I.; Ozler, K. B.; Ekenel, H. K.; Chowdhury, D. Paul; Bakshi, S.; Sa, P. K.; Majhni, B.; Peer, P.; Štruc, V. The Unconstrained Ear Recognition Challenge 2019 Proceedings Article V: International Conference on Biometrics (ICB 2019), 2019. Povzetek | Povezava | BibTeX | Oznake: biometrics, ear, ear recognitoin, uerc 2019 @inproceedings{emervsivc2019unconstrained, This paper presents a summary of the 2019 Unconstrained Ear Recognition Challenge (UERC), the second in a series of group benchmarking efforts centered around the problem of person recognition from ear images captured in uncontrolled settings. The goal of the challenge is to assess the performance of existing ear recognition techniques on a challenging large-scale ear dataset and to analyze performance of the technology from various viewpoints, such as generalization abilities to unseen data characteristics, sensitivity to rotations, occlusions and image resolution and performance bias on sub-groups of subjects, selected based on demographic criteria, i.e. gender and ethnicity. Research groups from 12 institutions entered the competition and submitted a total of 13 recognition approaches ranging from descriptor-based methods to deep-learning models. The majority of submissions focused on ensemble based methods combining either representations from multiple deep models or hand-crafted with learned image descriptors. Our analysis shows that methods incorporating deep learning models clearly outperform techniques relying solely on hand-crafted descriptors, even though both groups of techniques exhibit similar behaviour when it comes to robustness to various covariates, such presence of occlusions, changes in (head) pose, or variability in image resolution. The results of the challenge also show that there has been considerable progress since the first UERC in 2017, but that there is still ample room for further research in this area. |
Ziga, Emersic; Janez, Krizaj; Vitomir, Struc; Peter, Peer Deep ear recognition pipeline Book Section V: Mahmoud, Hassaballah; M., Hosny Khalid (Ur.): Recent advances in computer vision : theories and applications, vol. 804, Springer, 2019, ISBN: 1860-9503. Povzetek | Povezava | BibTeX | Oznake: ear, ear recognition, pipeline @incollection{ZigaBook2019, Ear recognition has seen multiple improvements in recent years and still remains very active today. However, it has been approached from recognition and detection perspective separately. Furthermore, deep-learning-based approaches that are popular in other domains have seen limited use in ear recognition and even more so in ear detection. Moreover, to obtain a usable recognition system a unified pipeline is needed. The input in such system should be plain images of subjects and the output identities based only on ear biometrics. We conduct separate analysis through detection and identification experiments on the challenging dataset and, using the best approaches, present a novel, unified pipeline. The pipeline is based on convolutional neural networks (CNN) and presents, to the best of our knowledge, the first CNN-based ear recognition pipeline. The pipeline incorporates both, the detection of ears on arbitrary images of people, as well as recognition on these segmented ear regions. The experiments show that the presented system is a state-of-the-art system and, thus, a good foundation for future real-word ear recognition systems. |
2018 |
Emeršič, Žiga; Gabriel, Luka; Štruc, Vitomir; Peer, Peter Convolutional encoder--decoder networks for pixel-wise ear detection and segmentation Članek v strokovni reviji V: IET Biometrics, vol. 7, no. 3, str. 175–184, 2018. Povzetek | Povezava | BibTeX | Oznake: annotated web ears, AWE, biometrics, ear, ear detection, pixel-wise detection, segmentation @article{emervsivc2018convolutional, Object detection and segmentation represents the basis for many tasks in computer and machine vision. In biometric recognition systems the detection of the region-of-interest (ROI) is one of the most crucial steps in the processing pipeline, significantly impacting the performance of the entire recognition system. Existing approaches to ear detection, are commonly susceptible to the presence of severe occlusions, ear accessories or variable illumination conditions and often deteriorate in their performance if applied on ear images captured in unconstrained settings. To address these shortcomings, we present a novel ear detection technique based on convolutional encoder-decoder networks (CEDs). We formulate the problem of ear detection as a two-class segmentation problem and design and train a CED-network architecture to distinguish between image-pixels belonging to the ear and the non-ear class. Unlike competing techniques, our approach does not simply return a bounding box around the detected ear, but provides detailed, pixel-wise information about the location of the ears in the image. Experiments on a dataset gathered from the web (a.k.a. in the wild) show that the proposed technique ensures good detection results in the presence of various covariate factors and significantly outperforms competing methods from the literature. |
2017 |
Emeršič, Žiga; Štepec, Dejan; Štruc, Vitomir; Peer, Peter Training convolutional neural networks with limited training data for ear recognition in the wild Proceedings Article V: IEEE International Conference on Automatic Face and Gesture Recognition, Workshop on Biometrics in the Wild 2017, 2017. Povzetek | Povezava | BibTeX | Oznake: CNN, convolutional neural networks, ear, ear recognition, limited data, model learning @inproceedings{emervsivc2017training, Identity recognition from ear images is an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes ear recognition technology an appealing choice for surveillance and security applications as well as related application domains. In contrast to other biometric modalities, where large datasets captured in uncontrolled settings are readily available, datasets of ear images are still limited in size and mostly of laboratory-like quality. As a consequence, ear recognition technology has not benefited yet from advances in deep learning and convolutional neural networks (CNNs) and is still lacking behind other modalities that experienced significant performance gains owing to deep recognition technology. In this paper we address this problem and aim at building a CNNbased ear recognition model. We explore different strategies towards model training with limited amounts of training data and show that by selecting an appropriate model architecture, using aggressive data augmentation and selective learning on existing (pre-trained) models, we are able to learn an effective CNN-based model using a little more than 1300 training images. The result of our work is the first CNN-based approach to ear recognition that is also made publicly available to the research community. With our model we are able to improve on the rank one recognition rate of the previous state-of-the-art by more than 25% on a challenging dataset of ear images captured from the web (a.k.a. in the wild). |
Emersic, Ziga; Meden, Blaz; Peer, Peter; Struc, Vitornir Covariate analysis of descriptor-based ear recognition techniques Proceedings Article V: 2017 international conference and workshop on bioinspired intelligence (IWOBI), str. 1–9, IEEE 2017. Povzetek | Povezava | BibTeX | Oznake: AWE, covariate analysis, descriptors, ear, performance evaluation @inproceedings{emersic2017covariate, Dense descriptor-based feature extraction techniques represent a popular choice for implementing biometric ear recognition system and are in general considered to be the current state-of-the-art in this area. In this paper, we study the impact of various factors (i.e., head rotation, presence of occlusions, gender and ethnicity) on the performance of 8 state-of-the-art descriptor-based ear recognition techniques. Our goal is to pinpoint weak points of the existing technology and identify open problems worth exploring in the future. We conduct our covariate analysis through identification experiments on the challenging AWE (Annotated Web Ears) dataset and report our findings. The results of our study show that high degrees of head movement and presence of accessories significantly impact the identification performance, whereas mild degrees of the listed factors and other covariates such as gender and ethnicity impact the identification performance only to a limited extent. |
Emeršič, Žiga; Štruc, Vitomir; Peer, Peter Ear recognition: More than a survey Članek v strokovni reviji V: Neurocomputing, vol. 255, str. 26–39, 2017. Povzetek | Povezava | BibTeX | Oznake: AWE, biometrics, dataset, ear, ear recognition, performance evalution, survey, toolbox @article{emervsivc2017ear, Automatic identity recognition from ear images represents an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes the technology an appealing choice for surveillance and security applications as well as other application domains. Significant contributions have been made in the field over recent years, but open research problems still remain and hinder a wider (commercial) deployment of the technology. This paper presents an overview of the field of automatic ear recognition (from 2D images) and focuses specifically on the most recent, descriptor-based methods proposed in this area. Open challenges are discussed and potential research directions are outlined with the goal of providing the reader with a point of reference for issues worth examining in the future. In addition to a comprehensive review on ear recognition technology, the paper also introduces a new, fully unconstrained dataset of ear images gathered from the web and a toolbox implementing several state-of-the-art techniques for ear recognition. The dataset and toolbox are meant to address some of the open issues in the field and are made publicly available to the research community. |
Objave
2023 |
The Unconstrained Ear Recognition Challenge 2023: Maximizing Performance and Minimizing Bias Proceedings Article V: IEEE International Joint Conference on Biometrics (IJCB 2023), str. 1-10, Ljubljana, Slovenia, 2023. |
Efficient ear alignment using a two-stack hourglass network Članek v strokovni reviji V: IET Biometrics , str. 1-14, 2023, ISSN: 2047-4938. |
2022 |
Kinship Verification from Ear Images: An Explorative Study with Deep Learning Models Proceedings Article V: International Workshop on Biometrics and Forensics (IWBF), str. 1–6, 2022. |
2019 |
The Unconstrained Ear Recognition Challenge 2019 Proceedings Article V: International Conference on Biometrics (ICB 2019), 2019. |
Deep ear recognition pipeline Book Section V: Mahmoud, Hassaballah; M., Hosny Khalid (Ur.): Recent advances in computer vision : theories and applications, vol. 804, Springer, 2019, ISBN: 1860-9503. |
2018 |
Convolutional encoder--decoder networks for pixel-wise ear detection and segmentation Članek v strokovni reviji V: IET Biometrics, vol. 7, no. 3, str. 175–184, 2018. |
2017 |
Training convolutional neural networks with limited training data for ear recognition in the wild Proceedings Article V: IEEE International Conference on Automatic Face and Gesture Recognition, Workshop on Biometrics in the Wild 2017, 2017. |
Covariate analysis of descriptor-based ear recognition techniques Proceedings Article V: 2017 international conference and workshop on bioinspired intelligence (IWOBI), str. 1–9, IEEE 2017. |
Ear recognition: More than a survey Članek v strokovni reviji V: Neurocomputing, vol. 255, str. 26–39, 2017. |