3D-GANTex:基於StyleGAN3的多視角圖像和3DDFA的網格生成方式重建3D人臉

Abstract

none

Texture estimation from a single image is a challenging task due to the lack of texture information available and limited training data. This thesis proposes a novel approach for texture estimation from a single in the wild image using a Generative Adversarial Network (GAN) and 3D Dense Face Alignment (3DDFA). The method begins by generating multi-view faces using the latent space of GAN. Then 3DDFA generates a 3D face mesh as well as a high-resolution texture map that is consistent with the estimated face shape. The generated texture map is later refined using an iterative process that incorporates information from both the input image and the estimated 3D face shape.Studies have been conducted to investigate the contributions of different components of the mentioned method, and show that: 1. Use of the GAN latent space can be a critical benchmark for achieving high-quality results. 2. Editing the latent space can generate high quality multi-view images. 3. Generating 3D mesh and texture map estimation from a single image is possible with a very high accuracy.

To evaluate the effectiveness of this approach, experiments were conducted on in-the-wild imagesand the results were compared with state of-the-art 3D Scanner. To verify that, subjective valuation has been performed on 16 participants. The results prove that the mentioned method outperforms existing method in terms of performance, demonstrating the effectiveness of this approach.Results generated from the aforementioned method are very accurate and has the potential to serve as an important contribution in avatar creation as well as 3D Face Reconstruction.

In summary, the proposed method for texture estimation from a single image using GAN latent space and 3DDFA represents a significant advancement in the field of computer vision and has potential applications in a wide range of fields, including virtual try-on, facial recognition, beauty industry as well as metaverse.