3D Hair Reconstruction Based on Hairstyle Attributes Learning from Single-view Hair Image Using Deep Learning

Abstract

Hair, as a vital component of the human's appearance, plays an important role in producing digital characters. However, the generation of realistic hairstyles usually needs professional digital artists and/or complex hardware, and the procedure is often time-consuming due to its numerous numbers, and diverse hairstyles. Thus, automatic capture of real-world hairstyles with easy input can greatly benefit the production pipeline. State-of-the-art 3D hair modeling systems require either multi-view images or a single- view image with complementary synthetic 3D hair models. For the multi-view image based 3D hair reconstruction, the capture systems are often made of a large number of cameras, projectors, light sources, and are usually in the indoor environment, which prevents popular use of the methods. On the contrary, single-view image based methods only use simple capture devices, e.g.; a handheld camera. However, a front view containing a face is often required and the resulting 3D hair strand reconstruction quality is compromised. Meanwhile, several hairstyles can not be easily modeled, such as braids and kinky hairstyles (afro-textured hairs), even though they are very common in real life. In this dissertation, we implement a single-view imaged based 3D hair modeling system, where our hair reconstruction is done through 2D hair analysis and 3D strands creation, which benefits from both traditional image processing techniques and the strengths of machine learning. Our 2D hair analysis is used to learn the attributes of input hairs, including 2D hair strands, detailed hairstyle patterns, and the corresponding parametric representation (which includes braids and kinky hairs), and braid structures. Simultaneously 3D hair strands are generated using deep-learning models. Our method is different from previous methods as our generated hair models can be modified by controlling the attributes and parameters we learned from the 2D hair recognition/analysis system. Our system does not require a face to be shown in the input image and to our best knowledge, our work is the first work that can reconstruct 3D braided hair and kinky hair given a single-view image. Qualitatively and quantitatively assessments indicate that our system can generate a variety of realistic 3D hairstyle models.