The Deep generative models for beauty and fashion (DeepBeauty) project aims to contribute to research on image generation and editing technology with a particular focus on deep learning methodologies, which have recently been shown to be a highly convenient and effective tool for this task. Our goal is to develop novel (exible and robust) mechanisms for image editing tailored towards the needs of the beauty and fashion industries, capable of altering certain parts of the input images in accordance with predefined target appearances (e.g., an example of specific makeup, an image of a model wearing a fashion item, clothing or accessory). The main tangible result of the project will be novel and highly robust virtual try-on technology based on original approaches to face and body editing. The developed technology, packaged in form of a demonstrator-engine, will be able to edit images in a photo-realistic manner, while preserving the overall visual appearance of the subjects in the images.
DeepBeauty (ARRS: J2-2501 (A)) is a fundamental research project funded by the Slovenian Research Agency (ARRS) in in the period: 1.9.2020 – 31.8.2023 (1,8 FTE per year).
The Principal Investigator (PI) of DeepBeauty is Assoc. Prof. Vitomir Štruc, PhD.
Link to SICRIS: Follow me.
DeepBeauty is structured into 6 work packages:
- WP1: Coordination and management
- WP2: Latent space exploration
- WP3: Face editing for beauty and fashion
- WP4: Virtual try-on
- WP5: Demo room and exploitation
- WP6: Dissemination
The R&D work on these work packages is expected to result in:
- A better understanding of the latent space of modern generative deep learning models
- Flexible and photo-realistic face editing technology
- Flexible and photo-realistic body editing technology
- Year 1: Activities on work packages WP1, WP2, WP3, WP4, WP6
- Year 2: Activities on work packages WP1, WP2, WP3, WP4, WP6
- Year 3: Activities on work packages WP1, WP2, WP3, WP4, WP5, WP6
DeepBeauty is conducted jointly by:
- The Laboratory for Machine Intelligence (LMI), Faculty of Electrical Engineering, University of Ljubljana
- The Computer Vision Laboratory (CVL), Faculty of Computer and Information Science, University of Ljubljana
- Alpineon Ltd.
- The Computer Vision Research Laboratory
- Prof. Vitomir Štruc, PhD – Project Leader (PI)
- Assoc. Prof. Simon Dobrišek, PhD, Researcher LMI
- Assoc. Prof. Janez Perš, PhD, Researcher LMI
- Ass. Janez Križaj, PhD, Resercher LMI
- Ass. Klemen Grm, Researcher LMI
- Ass. Martin Pernuš, Researcher LMI
- Ass. Marija Ivanovska, Researcher LMI
- Peter Rot, Researcher LMI
- Benjamin Fele, Researcher LMI
- Prof. Peter Peer, PhD, PI at CVL
- Prof. Franc Solina, PhD, Researcher CVL
- Ass. Prof. Žiga Emeršič, Researcher CVL
- Ass. Blaž Meden, Researcher CVL
- Matej Vitek, Researcher CVL
- Ajda Lampe, Researcher at CVL and LMI
- Jerneja Žganec Gros, PhD, PI at Alpineon
- Jaka Kravanja, PhD, Researcher Alpineon
International Advisory Committee
- Prof. Clinton Fookes, PhD, Queensland University of Technology, Australia
- Assoc. Prof. Hugo Pedro Proenca, Universidade da Beira Interior, Portugal
- Assoc. Prof. Vishal Patel, John Hopkins University
- Assoc. Prof. Walter Scheirer, PhD, University of Notre Dame, USA
- Martin Pernuš, Clinton Fookes, Vitomir Štruc, Simon Dobrišek, FICE: Text-Conditioned Fashion Image Editing with Guided GAN Inversion, arXiv preprint, 2022. [GitHub page]
- Richard Plesh, Peter Peer, Vitomir Štruc, GlassesGAN: Eyewear Personalization using Synthetic Appearance Discovery and Targeted Subspace Modeling, arXiv preprint, 2022 [PDF]
- Julijan Jug; Ajda Lampe; Vitomir Štruc; Peter Peer, Body Segmentation Using Multi-task Learning, International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 1-7, 2022 [PDF]
- Julijan Jug, Ajda Lampe, Peter Peer, Vitomir Štruc. Segmentacija telesa z uporabo večciljnega učenja, ROSUS, 2022 [PDF]
- Darian Tomašević, Peter Peer, Vitomir Štruc, BiOcularGAN: Bimodal Synthesis and Annotation of Ocular Images, IEEE/IAPR International Joint Conference on Biometrics (IJCB), 2022 [PDF][GitHub project]
- Benjamin Fele; Ajda Lampe; Peter Peer; Vitomir Štruc, C-VTON: Context-Driven Image-Based Virtual Try-On Network, IEEE/CVF Winter Applications in Computer Vision (WACV), pp. 1–10, 2022 [PDF][GitHub project]
- Martin Pernuš, Vitomir Štruc, Simon Dobrišek, High Resolution Face Editing with Masked GAN Latent Code Optimization, arXiv preprint, 2021 [PDF][GitHub page]
Open Source Code
- Martin Pernuš, Vitomir Štruc, Simon Dobrišek, Tomaž Černe, Jerneja Žganec-Gros, Postopek za napredno urejanje lastnosti obraza v digitalnih slikah, Patent application (approved), 2021 [PDF]
- Andraž Puc, Virtualno pomerjanje frizur z uporabo generativnih nevronskih modelov, Master Thesis, Supervisor: Vitomir Štruc, 2022 [PDF]
- Julijan Jug, Body segmentation using multi-task learning, Master thesis, Co-supervisors: Peter Peer and Vitomir Štruc, 2021 [PDF]
- Luka Zornada, Razvoj aplikacije za virtualno pomerjanje ličil in preurejanje obrazov, Bachelor Thesis, Supervisor: Vitomir Štruc, 2022 [PDF]
- Marija Jakimovska, Analiza modelov za preurejanje obraznih slik v lepotni industriji, Bachelor Thesis, Supervisor: Vitomir Štruc, 2022 [PDF]
- Vitomir Štruc, Computer vision for fashion and beauty, Keynote talk, Huawei Future Device Technology Summit, Helsinki, Finland, 8-9.11.2022. [PDF]
- Vitomir Štruc, Generative models in computer vision and biometrics, Lecture at the EURASIP JIVP Webinar, online, 6.10.2022 [video]
- Vitomir Štruc, Generative models in computer vision and biometrics, Lecture at Istanbul Technical University, Istanbul, Turkey, 21. 9. 2022.
- Vitomir Štruc, Photorealistic face editing via latent code optimization, Keynote talk, Workshop Digital Face Manipulation & Detection, organized by European Association for Biometrics (EAB), online, 12. 7. 2022
- Vitomir Štruc. Generative models in computer vision and biometrics, Lecture at the University of Salzburg, Salzburg, Austria, 14. 6. 2022.
- Vitomir Štruc, Generative models in biometrics, Talk at the Trustworthy Biometrics Webinar, IEEE Biometrics Beijing Chapter, online, 6. 5. 2022 [PDF]
- Vitomir Štruc, Generative Models for Computer Vision, Keynote Talk, Sixth IAPR International Conference on Computer Vision & Image Processing (CVIP2021), Punjab, India [PDF slides]
- Peter Peer. From ear recognition to deidentification and virtual try on, Lecture at Faculdade de Engenharia, Universidade do Porto, September 29, 2021.
- Benjamin Fele and Ajda Lampe, Award and recognition for research work conducted by PhD students for their WACV 2022 paper, Faculty of Computer and Information Science, UL, 2022.