Graphics Reference
In-Depth Information
noise and missing data, which makes it hard to establish the point-to-point correspondences,
and (3) it should be fully automatic. To establish point-to-point correspondences between
the 3D morphable face model and new face data with noise and missing data, we apply
our model-fitting method described in Section 4.3. This method either fits the model as a
single component or as a set of predefined components. In case the model is fitted as a single
component, the final model fit is in full point-to-point correspondence with the face model, but
adds no additional information to the current face model. In case the model is fitted as a set of
predefined components, this method produces model fits that go beyond the current statistics
of the face model, but the point-to-point correspondence are inaccurate or lost. In this section,
we briefly describe the used model-fitting method, then we explain our algorithm to establish
dense point-to-point correspondences between the multiple component fits and the morphable
face model, and finally, we explain how the bootstrapping algorithm can distinguish between
new face data to add to the model and redundant data to reject.
Model Fitting
The morphable face model that we use (see Section 4.4) has
m
99 coefficients that can be
varied to fit the face model to new scan data. To fit the model, we bring a new 3D face scan into
alignment with the 3D face model automatically. First, we automatically normalize the pose of
the face, detect the tip of the nose, and segment the face as described in Section 4.2. Secondly,
we align the face scan to the face model, coarsely by using their nose tips and more accurately
using the ICP algorithm. Then we apply the model-fitting algorithm as described in Section
4.3, using a set of four face components. This model-fitting algorithm iteratively adjusts the
m
coefficients
w
i
for each component, such that the vertices move closer to the vertices of the scan
data. After all components are fitted to the scan data individually, an accurate representation
of the new face
S
fine
is acquired. Each component can then be described using the set of
m
coefficients
w
i
for the eigenvectors of the face model. However, the fitted face instance
S
fine
may show artifacts at the borders of these fitted component. Note that we use the same PCA
model for each component, but with a subset of the model's vertices only.
=
Correspondence Estimation
After the application of the model-fitting method, most of the face model's vertices are brought
into correspondence with the face scan, but at the component's borders these point-to-point
correspondences are less accurate. Only in highly exceptional cases, borders are good enough
to bootstrap the face model with
S
fine
directly. To resolve the artifacts at the borders of the
individually fitted components, we use their sets of
m
coefficients
w
i
that were used to obtain
each component. In fact, each set of coefficients can be used to acquire a full face instance of
which the component is simply a predefined subset of vertices. We refer to such a full face
instance as
S
comp
. Because we fitted a set of
c
4 face instances
S
comp
. So the face instance
S
fine
is basically a composition of the
c
intermediate face instances
(Fig. 4.6). To blend the
c
components, we blend the vertices of the
c
face instances. In this
process, the goal is to determine for each vertex in
S
fine
a new position, such that it has a
smooth transition to its neighboring vertices. Once we reach this state, we refer to this final
face instance as
S
final
.
=
4 components, we have
c
=
