Deep generative models for beauty and fashion (DeepBeauty)

About

The Deep generative models for beauty and fashion (DeepBeauty) project aims to contribute to research on image generation and editing technology with a particular focus on deep learning methodologies, which have recently been shown to be a highly convenient and effective tool for this task. Our goal is to develop novel (exible and robust) mechanisms for image editing tailored towards the needs of the beauty and fashion industries, capable of altering certain parts of the input images in accordance with predefined target appearances (e.g., an example of specific makeup, an image of a model wearing a fashion item, clothing or accessory). The main tangible result of the project will be novel and highly robust virtual try-on technology based on original approaches to face and body editing. The developed technology, packaged in form of a demonstrator-engine, will be able to edit images in a photo-realistic manner, while preserving the overall visual appearance of the subjects in the images.

DeepBeauty is a fundamental research project funded by the Slovenian Research Agency (ARRS) in in the period: 1.9.2020 – 31.8.2023 (1,8 FTE per year).

The Principal Investigator (PI) of DeepBeauty is Assoc. Prof. Vitomir Štruc, PhD.

Link to SICRIS: Follow me.


Project overview

DeepBeauty is structured into 6 work packages:

  • WP1: Coordination and management
  • WP2: Latent space exploration
  • WP3: Face editing for beauty and fashion
  • WP4: Virtual try-on
  • WP5: Demo room and exploitation
  • WP6: Dissemination

The R&D work on these work packages is expected to result in:

  • A better understanding of the latent space of modern generative deep learning models
  • Flexible and photo-realistic face editing technology
  • Flexible and photo-realistic body editing technology

Project phases

  • Year 1: Activities on work packages WP1, WP2, WP3, WP4, WP6
  • Year 2: Activities on work packages WP1, WP2, WP3, WP4, WP6
  • Year 3: Activities on work packages WP1, WP2, WP3, WP4, WP5, WP6

Partners

DeepBeauty is conducted jointly by:


Participating researchers

International Advisory Committee


Project publications

Julijan Jug; Ajda Lampe; Vitomir Štruc; Peter Peer, Body Segmentation Using Multi-task Learning, International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 1-7, 2022 [PDF]

Benjamin Fele; Ajda Lampe; Peter Peer; Vitomir Štruc, C-VTON: Context-Driven Image-Based Virtual Try-On Network, IEEE/CVF Winter Applications in Computer Vision (WACV), pp. 1–10, 2022 [PDF][GitHub project]

Martin Pernuš, Vitomir Štruc, Simon Dobrišek, High Resolution Face Editing with Masked GAN Latent Code Optimization, arXiv preprint, 2021 [PDF]


Master Theses

Julijan Jug, Body segmentation using multi-task learning, Master thesis, Co-supervisors: Peteer Peer and Vitomir Štruc, 2021 [PDF]


Invited Talks

Vitomir Štruc, Generative Models for Computer Vision, Keynote Talk, Sixth IAPR International Conference on Computer Vision & Image Processing (CVIP2021), Punjab, India [PDF slides]

Peter Peer. From ear recognition to deidentification and virtual try on, Lecture at Faculdade de Engenharia, Universidade do Porto, September 29, 2021.


Funding agency