3D Face Modeling - 3D Face Modeling, Analysis and Recognition - page 13

Figure 1.7 Sample results on 3D modeling algorithm for calibrated multiview stereopsis proposed by
Furukawa and Ponce (2010) that outputs a quasi-dense set of rectangular patches covering the surfaces
visible in the input images. In each case, one of the input images is shown on the left, along with two views
of textured-mapped reconstructed patches and shaded polygonal surfaces. Copyright C 2007, IEEE

and ordering constraints are checked, and the pixels that do not fulfill these criteria are reached
using the disparity estimated at neighboring pixels. The limited search area ensures smoothness
and ordering constraints, but the uniqueness constraint is enforced again by disparity map
refinement. The refinement is defined as a linear combination of a photometric consistency
term, d p , and a surface consistency term, d s , balanced both by a user-specified smoothness
parameter, w s , and a data-driven parameter, w p , to ensure that the photometric term has the
greatest weight in regions with good feature localization. d p favors solutions with high NCC,
whereas d s favors smooth solutions. The refinement is performed on the disparity map and
later on the surface. Both are implemented as iterative processes.
The refinement results in surface geometry that is smooth across skin pores and fine wrinkles
because the disparity change across such a feature is too small to detect. The result is flatness
and lack of realism in synthesized views of the face. On the other hand, visual inspection
shows the obvious presence of pores and fine wrinkles in the images. This is due to the fact
that light reflected by a diffuse surface is related to the integral of the incoming light. In small
concavities, such as pores, part of the incoming light is blocked and the point thus appears
darker. This has been exploited by various authors (e.g., Glencross et al., 2008)) to infer local
geometry variation. In this section, we expose a method to embed this observation into the
surface refinement framework. It should be noticed that this refinement is qualitative, and the
geometry that is recovered is not metrically correct. However, augmenting the macroscopic
geometry with fine scale features does produce a significant improvement in the perceived
quality of the reconstructed face geometry.
For the mesoscopic augmentation, only features that are too small to be recovered by the
stereo algorithm are interesting. Therefore, first high pass filtered values are computed for all
points X using the projection of a Gaussian N :

c ν α c I c (X)

− N c I c (X)

c ν α c

μ (X) = (1.13)