Results of existing strand-based 3D reconstruction methods and our method tested with the data captured by a multi-camera system. Dr.Hair demonstrates better precision in reconstructing the directional flow of scalp-connected hair.
In the film and gaming industries, achieving a realistic hair appearance typically involves the use of strands originating from the scalp. However, reconstructing these strands from observed surface images of hair presents significant challenges. The difficulty in acquiring Ground Truth (GT) data has led state-of-the-art learning-based methods to rely on pre-training with manually prepared synthetic CG data. This process is not only labor-intensive and costly but also introduces complications due to the domain gap when compared to real-world data. In this study, we propose an optimization-based approach that eliminates the need for pre-training. Our method represents hair strands as line segments growing from the scalp and optimizes them using a novel differentiable rendering algorithm. To robustly optimize a substantial number of slender explicit geometries, we introduce 3D orientation estimation utilizing global optimization, strand initialization based on Laplace's equation, and reparameterization that leverages geometric connectivity and spatial proximity. Unlike existing optimization-based methods, our method is capable of reconstructing internal hair flow in an absolute direction. Our method exhibits robust and accurate inverse rendering, surpassing the quality of existing methods and significantly improving processing speed.
Our approach combines traditional real-time rendering techniques with recent advances in differentiable rendering. First, we fit a template to a raw mesh. Next, we compute consistent 3D orientations from 2D orientation images and initialize guide strands based on a differential equation. Finally,optimization based on differentiable rendering is applied by leveraging the hierarchical relationship between guides and children.
@InProceedings{takimoto2024drhair,
title = {Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-training via Differentiable Rendering of Line Segments},
author = {Takimoto, Yusuke and Takehara, Hikari and Sato, Hiroyuki and Zhu, Zihao and Zheng, Bo},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2024}
}