Objave

2021
Grm, Klemen; Vitomir, Štruc Frequency Band Encoding for Face Super-Resolution Proceedings Article V: Proceedings of ERK 2021, str. 1-4, 2021. Povzetek \| Povezava \| BibTeX \| Oznake: CNN, deep learning, face, face hallucination, frequency encoding, super-resolution @inproceedings{Grm-SuperResolution_ERK2021, title = {Frequency Band Encoding for Face Super-Resolution}, author = {Klemen Grm and Štruc Vitomir}, url = {https://lmi.fe.uni-lj.si/wp-content/uploads/2021/10/SRAE_ERK21.pdf}, year = {2021}, date = {2021-09-10}, booktitle = {Proceedings of ERK 2021}, pages = {1-4}, abstract = {In this paper, we present a novel method for face super-resolution based on an encoder-decoder architecture. Unlike previous approaches, which focused primarily on directly reconstructing the high-resolution face appearance from low-resolution images, our method relies on a multi-stage approach where we learn a face representation in different frequency bands, followed by decoding the representation into a high-resolution image. Using quantitative experiments, we are able to demonstrate that this approach results in better face image reconstruction, as well as aiding in downstream semantic tasks such as face recognition and face verification.}, keywords = {CNN, deep learning, face, face hallucination, frequency encoding, super-resolution}, pubstate = {published}, tppubtype = {inproceedings} } Zapri In this paper, we present a novel method for face super-resolution based on an encoder-decoder architecture. Unlike previous approaches, which focused primarily on directly reconstructing the high-resolution face appearance from low-resolution images, our method relies on a multi-stage approach where we learn a face representation in different frequency bands, followed by decoding the representation into a high-resolution image. Using quantitative experiments, we are able to demonstrate that this approach results in better face image reconstruction, as well as aiding in downstream semantic tasks such as face recognition and face verification. Zapri https://lmi.fe.uni-lj.si/wp-content/uploads/2021/10/SRAE_ERK21.pdf Zapri
2020
Grm, Klemen; Scheirer, Walter J.; Štruc, Vitomir Face hallucination using cascaded super-resolution and identity priors Članek v strokovni reviji V: IEEE Transactions on Image Processing, 2020. Povzetek \| Povezava \| BibTeX \| Oznake: biometrics, CNN, computer vision, deep learning, face, face hallucination, super-resolution @article{TIPKlemen_2020, title = {Face hallucination using cascaded super-resolution and identity priors}, author = {Klemen Grm and Walter J. Scheirer and Vitomir Štruc}, url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8866753 https://lmi.fe.uni-lj.si/wp-content/uploads/2023/02/IEEET_face_hallucination_compressed.pdf}, doi = {10.1109/TIP.2019.2945835}, year = {2020}, date = {2020-01-01}, urldate = {2020-01-01}, journal = {IEEE Transactions on Image Processing}, abstract = {In this paper we address the problem of hallucinating high-resolution facial images from low-resolution inputs at high magnification factors. We approach this task with convolutional neural networks (CNNs) and propose a novel (deep) face hallucination model that incorporates identity priors into the learning procedure. The model consists of two main parts: i) a cascaded super-resolution network that upscales the lowresolution facial images, and ii) an ensemble of face recognition models that act as identity priors for the super-resolution network during training. Different from most competing super-resolution techniques that rely on a single model for upscaling (even with large magnification factors), our network uses a cascade of multiple SR models that progressively upscale the low-resolution images using steps of 2×. This characteristic allows us to apply supervision signals (target appearances) at different resolutions and incorporate identity constraints at multiple-scales. The proposed C-SRIP model (Cascaded Super Resolution with Identity Priors) is able to upscale (tiny) low-resolution images captured in unconstrained conditions and produce visually convincing results for diverse low-resolution inputs. We rigorously evaluate the proposed model on the Labeled Faces in the Wild (LFW), Helen and CelebA datasets and report superior performance compared to the existing state-of-the-art. }, keywords = {biometrics, CNN, computer vision, deep learning, face, face hallucination, super-resolution}, pubstate = {published}, tppubtype = {article} } Zapri In this paper we address the problem of hallucinating high-resolution facial images from low-resolution inputs at high magnification factors. We approach this task with convolutional neural networks (CNNs) and propose a novel (deep) face hallucination model that incorporates identity priors into the learning procedure. The model consists of two main parts: i) a cascaded super-resolution network that upscales the lowresolution facial images, and ii) an ensemble of face recognition models that act as identity priors for the super-resolution network during training. Different from most competing super-resolution techniques that rely on a single model for upscaling (even with large magnification factors), our network uses a cascade of multiple SR models that progressively upscale the low-resolution images using steps of 2×. This characteristic allows us to apply supervision signals (target appearances) at different resolutions and incorporate identity constraints at multiple-scales. The proposed C-SRIP model (Cascaded Super Resolution with Identity Priors) is able to upscale (tiny) low-resolution images captured in unconstrained conditions and produce visually convincing results for diverse low-resolution inputs. We rigorously evaluate the proposed model on the Labeled Faces in the Wild (LFW), Helen and CelebA datasets and report superior performance compared to the existing state-of-the-art. Zapri https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8866753 https://lmi.fe.uni-lj.si/wp-content/uploads/2023/02/IEEET_face_hallucination_com[...] doi:10.1109/TIP.2019.2945835 Zapri
2019
Grm, Klemen; Pernus, Martin; Cluzel, Leo; Scheirer, Walter J.; Dobrisek, Simon; Struc, Vitomir Face Hallucination Revisited: An Exploratory Study on Dataset Bias Proceedings Article V: IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019. Povzetek \| Povezava \| BibTeX \| Oznake: dataset bias, face, face hallucination, super-resolution @inproceedings{grm2019face, title = {Face Hallucination Revisited: An Exploratory Study on Dataset Bias}, author = {Klemen Grm and Martin Pernus and Leo Cluzel and Walter J. Scheirer and Simon Dobrisek and Vitomir Struc}, url = {http://openaccess.thecvf.com/content_CVPRW_2019/papers/Biometrics/Grm_Face_Hallucination_Revisited_An_Exploratory_Study_on_Dataset_Bias_CVPRW_2019_paper.pdf https://arxiv.org/pdf/1812.09010.pdf}, year = {2019}, date = {2019-06-01}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition Workshops}, abstract = {Contemporary face hallucination (FH) models exhibit considerable ability to reconstruct high-resolution (HR) details from low-resolution (LR) face images. This ability is commonly learned from examples of corresponding HR-LR image pairs, created by artificially down-sampling the HR ground truth data. This down-sampling (or degradation) procedure not only defines the characteristics of the LR training data, but also determines the type of image degradations the learned FH models are eventually able to handle. If the image characteristics encountered with real-world LR images differ from the ones seen during training, FH models are still expected to perform well, but in practice may not produce the desired results. In this paper we study this problem and explore the bias introduced into FH models by the characteristics of the training data. We systematically analyze the generalization capabilities of several FH models in various scenarios where the degradation function does not match the training setup and conduct experiments with synthetically downgraded as well as real-life low-quality images. We make several interesting findings that provide insight into existing problems with FH models and point to future research directions.}, keywords = {dataset bias, face, face hallucination, super-resolution}, pubstate = {published}, tppubtype = {inproceedings} } Zapri Contemporary face hallucination (FH) models exhibit considerable ability to reconstruct high-resolution (HR) details from low-resolution (LR) face images. This ability is commonly learned from examples of corresponding HR-LR image pairs, created by artificially down-sampling the HR ground truth data. This down-sampling (or degradation) procedure not only defines the characteristics of the LR training data, but also determines the type of image degradations the learned FH models are eventually able to handle. If the image characteristics encountered with real-world LR images differ from the ones seen during training, FH models are still expected to perform well, but in practice may not produce the desired results. In this paper we study this problem and explore the bias introduced into FH models by the characteristics of the training data. We systematically analyze the generalization capabilities of several FH models in various scenarios where the degradation function does not match the training setup and conduct experiments with synthetically downgraded as well as real-life low-quality images. We make several interesting findings that provide insight into existing problems with FH models and point to future research directions. Zapri http://openaccess.thecvf.com/content_CVPRW_2019/papers/Biometrics/Grm_Face_Hallu[...] https://arxiv.org/pdf/1812.09010.pdf Zapri
2017
Klemen, Grm; Simon, Dobrišek; Vitomir, Štruc Evaluating image superresolution algorithms for cross-resolution face recognition Proceedings Article V: Proceedings of the Twenty-sixth International Electrotechnical and Computer Science Conference ERK 2017, 2017. Povzetek \| Povezava \| BibTeX \| Oznake: face, face hallucination, face recognition, performance evaluation, super-resolution @inproceedings{ERK2017Grm, title = {Evaluating image superresolution algorithms for cross-resolution face recognition}, author = {Grm Klemen and Dobrišek Simon and Štruc Vitomir}, url = {https://lmi.fe.uni-lj.si/wp-content/uploads/2019/08/review_submission.pdf}, year = {2017}, date = {2017-09-01}, booktitle = {Proceedings of the Twenty-sixth International Electrotechnical and Computer Science Conference ERK 2017}, abstract = {With recent advancements in deep learning and convolutional neural networks (CNNs), face recognition has seen significant performance improvements over the last few years. However, low-resolution images still remain challenging, with CNNs performing relatively poorly compared to humans. One possibility to improve performance in these settings often advocated in the literature is the use of super-resolution (SR). In this paper, we explore the usefulness of SR algorithms for cross-resolution face recognition in experiments on the Labeled Faces in the Wild (LFW) and SCface datasets using four recent deep CNN models. We conduct experiments with synthetically down-sampled images as well as real-life low-resolution imagery captured by surveillance cameras. Our experiments show that image super-resolution can improve face recognition performance considerably on very low-resolution images (of size 24 x 24 or 32 x 32 pixels), when images are artificially down-sampled, but has a lesser (or sometimes even a detrimental) effect with real-life images leaving significant room for further research in this area.}, keywords = {face, face hallucination, face recognition, performance evaluation, super-resolution}, pubstate = {published}, tppubtype = {inproceedings} } Zapri With recent advancements in deep learning and convolutional neural networks (CNNs), face recognition has seen significant performance improvements over the last few years. However, low-resolution images still remain challenging, with CNNs performing relatively poorly compared to humans. One possibility to improve performance in these settings often advocated in the literature is the use of super-resolution (SR). In this paper, we explore the usefulness of SR algorithms for cross-resolution face recognition in experiments on the Labeled Faces in the Wild (LFW) and SCface datasets using four recent deep CNN models. We conduct experiments with synthetically down-sampled images as well as real-life low-resolution imagery captured by surveillance cameras. Our experiments show that image super-resolution can improve face recognition performance considerably on very low-resolution images (of size 24 x 24 or 32 x 32 pixels), when images are artificially down-sampled, but has a lesser (or sometimes even a detrimental) effect with real-life images leaving significant room for further research in this area. Zapri https://lmi.fe.uni-lj.si/wp-content/uploads/2019/08/review_submission.pdf Zapri