Modeling and Synthesis of Realistic Visual Speech in 3D - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 278

The virtual displacements d j of equation (1) are determined by demanding that
the 116 vertices undergo the (known) motions that map them from the generic
model onto the 3D face patch. This condition leads to a system of equations for
the X , Y , and Z components of the 116 vertex motions, which are combined into
three column vectors c X, Y, Z respectively:

Ad = c

. (5)

XYZ ,, XYZ ,,

In these equations, the vectors d X, Y, Z represent the column vectors containing all
the X , Y , or Z components of the virtual displacement vectors d j . The influence
matrix A contains the weights that the vertices of the known network apply to
each other. After solving these systems for c X, Y, Z , the interpolation is ready to be
applied. It is important to note that vertices on different sides of the dividing line
of the mouth are decoupled in these calculations.
A third step in the processing projects the interpolated points onto the extracted
3D surface. This is achieved via a cylindrical mapping. This mapping is not
carried out for a small subset of points which lie in a cavity, however. The reason
is that the acquisition system does not always produce good data in these cavities.
The position of these points should be determined fully by the deformed head
model, and not subject to being degraded under the influence of the acquired
data. They are shown on the right side of Figure 8. On Figure 8(3), this is
illustrated for the nostril. The extracted 3D grid is too smooth there and does not
follow the sharp dip that the nose takes. The generic model dominates the fitting
procedure and caters for the desired, high curvatures, as can be seen.

Figure 9. The jaw and lower teeth rotate around the midpoint of the places
where the jaw is attached to the skull, and translated (see text).