GEOMETRIC FACIAL MOTION SYNTHESIS - 3D Face Processing: Modeling, Analysis and Synthesis - page 38

Chapter 5

GEOMETRIC FACIAL MOTION SYNTHESIS

In this section, we discuss the facial motion synthesis using the trained Mo-
tion Unit model. Our goal is to use learned models to synthesize plausible facial
appearance for providing visual cues in synthetic face based interactions. In
Chapter 3, we have discussed that linear combinations of geometric MUs can be
used to approximate arbitrary face shapes. In this chapter, we focus on how to
produce animation according to various input signals, such as text and speech.
We first review previous work on face animation in Section 1. Next, we intro-
duce face motion trajectory for face animation in Section 2. After text-driven
face animation and offline speech-driven animation are discussed in Section 3
and 4, respectively. Finally we describe real-time speech-driven animation in
Section 5.

1. Previous Work

Based on spatial and temporal modeling of facial deformation, facial mo-
tion is usually synthesized according to semantic input, such as actor perfor-
mance [Williams, 1990], text script [Waters and Levergood, 1993], or speech [Brand,
1999, Morishima and Yotsukura, 1999].

1.1 Performance-driven face animation

Performance-driven face animation animates face models according to vi-
sual input signals. This type of approach automatically analyzes real facial
movements in the video using computer vision techniques. The analysis re-
sults are used to animate graphic face models. Williams [Williams, 1990] put
markers on the subject's face and use simple tracking algorithm to estimate the
motion of the markers. Guenter et al. [Guenter et al., 1998] put more markers
on faces and track their 3D motion to achieve high quality visual input. How-