Learning 3D Face Deformation Model: Methods and Applications - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 326

Figure 5. (a) NMF learned parts overlayed on the generic face model; (b)
The facial muscle distribution; (c) The aligned facial muscle distribution;
(d) The parts overlayed on muscle distribution; (e) The final parts.

(a) (b) (c) (d) (e)

can thus be: (1) more related to meaningful facial muscle distribution; and (2) less
biased by individuality in the motion capture data and, thus, more easily
generalized to different faces. We start with an image of human facial muscle,
illustrated in Figure 5(b) (Facial muscle image, 2002). Next, we align it with our
generic face model via image warping, based on facial feature points illustrated
in Figure 2(c). The aligned facial muscle image is shown in Figure 5(c). Then,
we overlay the learned parts on facial muscle distribution (Figure 5(d)) and
interactively adjust the learned parts such that different parts correspond to
different muscles. The final parts are shown in Figure 5(e).
The learned parts-based MUs give more flexibility in local facial deformation
analysis and synthesis. Figure 6 shows some local deformation in the lower lips,
each of which is induced by one of the learned parts-based MUs. These locally
deformed shapes are difficult to approximate using holistic MUs. For each local
deformation shown in Figure 6, more than 100 holistic MUs are needed to
achieve 90% reconstruction accuracy. That means, although some local defor-
mation is induced by only one parts-based MU, more than 100 holistic MUs may
be needed in order to achieve good analysis and synthesis quality. Therefore, we
can have more flexibility in using parts-based MUs. For example, if we are only
interested in lip motion, we only need to learn parts-based MUs from lip motion
data. In face animation, people often want to animate local regions separately.
This task can be easily achieved by adjusting the MUPs of parts-based MUs
separately. In face tracking, people may use parts-based MUs to track only the
region of their interests (e.g., the lips). Furthermore, tracking using parts-based
MUs is more robust because local error will not affect distant regions.