2026 |
Todorov, Leon; Rot, Peter; Peer, Peter; Štruc, Vitomir; Grm, Klemen SCOUT: Semantic Concept Discovery for Open-Vocabulary Editing of Face Recognition Templates Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB 2026), str. 1–10, 2026. Povzetek | Povezava | BibTeX | Oznake: face analysis, face generation, face images, face recognition, xai @inproceedings{LeonIJCB2026,Face recognition templates are compact identity representations, yet they also encode rich semantic information about facial appearance. Prior work has shown that templates can be inverted to images or indirectly manipulated through image-editing pipelines, but direct semantic editing in template space remains largely unexplored. Existing interpretability methods for face recognition often rely on manual neuron inspection or predefined attribute labels, limiting scalability and semantic flexibility. To address this gap, we propose SCOUT (Semantic Concept Discovery for Open-VocabUlary Editing of Face Recognition Templates), an end-to-end framework for discovering and directly manipulating semantic concepts in face recognition templates using mechanistic interpretability. SCOUT learns sparse template representations, generates semantic hypotheses for latent features from natural-language descriptions, and validates their stability. The resulting features act as controllable semantic directions for direct editing, avoiding costly edit--re-encode pipelines. Experiments with face recognition models using CNN, ViT, and Swin backbones show that SCOUT discovers interpretable concepts beyond standard attribute labels and enables controllable, identity-aware template manipulation with negligible impact on identity matching. We further show that edited templates can subsequently be decoded with independent inversion models for visualization and evaluation. |
Estrada, Jose Edgar Hernandez Cancino; Diaz, Mauro; Emeršič, Žiga; Štruc, Vitomir; Peer, Peter; Tomašević, Darian PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB 2026), str. 1–10, 2026. Povzetek | Povezava | BibTeX | Oznake: face generation, generative AI, unlearning @inproceedings{Unlearning2026,Identity-conditioned diffusion models enable high-quality and identity-consistent face generation, but they also raise severe privacy concerns, as models may continue to synthesize individuals despite their right to be forgotten. While machine unlearning has been extensively studied for concept and data removal, identity unlearning remains largely unexplored, particularly in models conditioned directly on identity embeddings rather than text prompts. In this work, we study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent diffusion model for face generation, and introduce textbf{P}roximity-guided textbf{I}dentity textbf{U}nlearning (textbf{PIU}), an anchor-guided framework for identity unlearning. Specifically, we formulate identity removal as an identity replacement objective that reassigns the source identity to a selected anchor identity in the learned identity space, and we complement it with a proximity-based anchor selection strategy motivated by the geometry of ArcFace representations. We further show that effective unlearning can be achieved through localized fine-tuning of a small subset of identity-sensitive cross-attention layers. Experiments across multiple target identities show that our framework effectively suppresses generation of the target identity while preserving realism and identity consistency for retained identities, as validated by improved performance on unlearning and image-quality metrics, together with qualitative evaluation. The source code for PIU is publicly available at url{https://github.com/edgarcancinoe/piu_unlearning} |
Sabadin, Jernej; Tomašević, Darian; Meden, Blaž; Peer, Peter; Štruc, Vitomir IDSync: Improving Diffusion Models Through Identity Classification Proceedings Article V: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (FG), str. 1–10, 2026. Povzetek | Povezava | BibTeX | Oznake: biometrics, data synthesis, face generation, face synthesis, generative AI, generative models @inproceedings{JernejFG2026,Effective training of face recognition models requires large-scale datasets of facial identities, yet collecting suitable data is time-consuming and raises privacy concerns. Existing deep generative models offer a promising alternative through the synthesis of high-quality images but often fail to fully preserve identity information. In this work, we propose IDSync, a novel generative diffusion-based framework designed to produce synthetic face images with more consistent identities that are better suited for training recognition models. To this end, IDSync employs a denoising network in the latent space of a frozen variational autoencoder, with identity guidance introduced via a text encoder that interprets identity embeddings from a pretrained recognition model. During training, the framework leverages a pretrained auxiliary identity classifier to define an additional cross-entropy loss, which is backpropagated to improve identity consistency. We evaluate the generated images using inter- and intra-class cosine similarity of identity features along with a variety of statistical measures between synthetic and real distributions focused on fidelity and diversity. To assess utility, we train face recognition models on the synthetic images and measure accuracy on standard verification benchmarks. Experimental results show that recognition models trained on IDSync-generated data achieve higher verification accuracies on real-world benchmarks than models trained on synthetic data produced by competing generative models. The IDSync source code is publicly available at url{https://github.com/JSabadin/IDSync}. |
2024 |
Tomašević, Darian; Boutros, Fadi; Damer, Naser; Peer, Peter; Štruc, Vitomir Generating bimodal privacy-preserving data for face recognition Članek v strokovni reviji V: Engineering Applications of Artificial Intelligence, vol. 133, iss. E, str. 1-25, 2024. Povzetek | Povezava | BibTeX | Oznake: CNN, face, face generation, face images, face recognition, generative AI, StyleGAN2, synthetic data @article{Darian2024,The performance of state-of-the-art face recognition systems depends crucially on the availability of large-scale training datasets. However, increasing privacy concerns nowadays accompany the collection and distribution of biometric data, which has already resulted in the retraction of valuable face recognition datasets. The use of synthetic data represents a potential solution, however, the generation of privacy-preserving facial images useful for training recognition models is still an open problem. Generative methods also remain bound to the visible spectrum, despite the benefits that multispectral data can provide. To address these issues, we present a novel identity-conditioned generative framework capable of producing large-scale recognition datasets of visible and near-infrared privacy-preserving face images. The framework relies on a novel identity-conditioned dual-branch style-based generative adversarial network to enable the synthesis of aligned high-quality samples of identities determined by features of a pretrained recognition model. In addition, the framework incorporates a novel filter to prevent samples of privacy-breaching identities from reaching the generated datasets and improve both identity separability and intra-identity diversity. Extensive experiments on six publicly available datasets reveal that our framework achieves competitive synthesis capabilities while preserving the privacy of real-world subjects. The synthesized datasets also facilitate training more powerful recognition models than datasets generated by competing methods or even small-scale real-world datasets. Employing both visible and near-infrared data for training also results in higher recognition accuracy on real-world visible spectrum benchmarks. Therefore, training with multispectral data could potentially improve existing recognition systems that utilize only the visible spectrum, without the need for additional sensors. |
2023 |
Pernuš, Martin; Bhatnagar, Mansi; Samad, Badr; Singh, Divyanshu; Peer, Peter; Štruc, Vitomir; Dobrišek, Simon ChildNet: Structural Kinship Face Synthesis Model With Appearance Control Mechanisms Članek v strokovni reviji V: IEEE Access, str. 1-22, 2023, ISSN: 2169-3536. Povzetek | Povezava | BibTeX | Oznake: artificial intelligence, CNN, deep learning, face generation, face synthesis, GAN, GAN inversion, kinship, kinship synthesis, StyleGAN2 @article{AccessMartin2023,Kinship face synthesis is an increasingly popular topic within the computer vision community, particularly the task of predicting the child appearance using parental images. Previous work has been limited in terms of model capacity and inadequate training data, which is comprised of low-resolution and tightly cropped images, leading to lower synthesis quality. In this paper, we propose ChildNet, a method for kinship face synthesis that leverages the facial image generation capabilities of a state-of-the-art Generative Adversarial Network (GAN), and resolves the aforementioned problems. ChildNet is designed within the GAN latent space and is able to predict a child appearance that bears high resemblance to real parents’ children. To ensure fine-grained control, we propose an age and gender manipulation module that allows precise manipulation of the child synthesis result. ChildNet is capable of generating multiple child images per parent pair input, while providing a way to control the image generation variability. Additionally, we introduce a mechanism to control the dominant parent image. Finally, to facilitate the task of kinship face synthesis, we introduce a new kinship dataset, called Next of Kin. This dataset contains 3690 high-resolution face images with a diverse range of ethnicities and ages. We evaluate ChildNet in comprehensive experiments against three competing kinship face synthesis models, using two kinship datasets. The experiments demonstrate the superior performance of ChildNet in terms of identity similarity, while exhibiting high perceptual image quality. The source code for the model is publicly available at: https://github.com/MartinPernus/ChildNet. |
Objave
2026 |
SCOUT: Semantic Concept Discovery for Open-Vocabulary Editing of Face Recognition Templates Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB 2026), str. 1–10, 2026. |
PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models Proceedings Article V: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB 2026), str. 1–10, 2026. |
IDSync: Improving Diffusion Models Through Identity Classification Proceedings Article V: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (FG), str. 1–10, 2026. |
2024 |
Generating bimodal privacy-preserving data for face recognition Članek v strokovni reviji V: Engineering Applications of Artificial Intelligence, vol. 133, iss. E, str. 1-25, 2024. |
2023 |
ChildNet: Structural Kinship Face Synthesis Model With Appearance Control Mechanisms Članek v strokovni reviji V: IEEE Access, str. 1-22, 2023, ISSN: 2169-3536. |