INTRODUCTION - 3D Face Processing: Modeling, Analysis and Synthesis - page 7

has a delay of only 100 ms, which will not interfere with real-time two-way
communications.
2.4 Enhanced facial motion analysis and synthesis using
flexible appearance model
Besides the geometric deformations modeled from motion capture data, fa-
cial motions also exhibit detailed appearance changes such as wrinkles and
creases as well. These details are important visual cues but they are difficult to
analyze and synthesize using geometric-based approaches. Appearance-based
models have been adopted to deal with this problem [Bartlett et al., 1999, Do-
nato et al., 1999]. Previous appearance-based approaches were mostly based on
extensive training appearance examples. However, the space of all face appear-
ance is huge, affected by the variations across different head poses, individuals,
lighting, expressions, speech and etc. Thus it is difficult for appearance-based
methods to collect enough face appearance data and train a model that works ro-
bustly in many different scenarios. In this respect, the geometric-feature-based
methods are more robust to large head motions, changes of lighting and are less
person-dependent.
To combine the advantages of both approaches, people have been investigat-
ing methods of using both geometry (shape) and appearance (texture) in face
analysis and synthesis. The Active Appearance Model (AAM) [Cootes et al.,
1998] and its variants, apply PCA to model both the shape variations of image
patches and their texture variations. They have been shown to be powerful
tools for face alignment, recognition, and synthesis. Blanz and Vetter [Blanz
and Vetter, 1999] propose 3D morphable models for 3D faces modeling, which
model the variations of both 3D face shape and texture using PCA. The 3D
morphable models have been shown effective in 3D face animation and face
recognition from non-frontal views [Blanz et al., 2002]. In facial expression
classification, Tian et al. [Tian et al., 2002] and Zhang et al. [Zhang et al.,
1998] propose to train classifiers (e.g. neural networks) using both shape and
texture features. The trained classifiers were shown to outperform classifiers
using shape or texture features only. In these approaches, some variations of
texture are absorbed by shape variation models. However, the potential texture
space can still be huge because many other variations are not modelled by shape
model. Moreover, little has been done to adapt the learned models to new con-
ditions. As a result, the application of these methods are limited to conditions
similar to those of training data.
In this topic, we propose a flexible appearance model in our framework to
deal with detailed facial motions. We have developed an efficient method for
modeling illumination effects from a single face image. We also apply ratio-
image technique [Liu et al., 200la] to reduce person-dependency in a principled
way. Using these two techniques, we design novel appearance features and use