INTRODUCTION - 3D Face Processing: Modeling, Analysis and Synthesis - page 3

Figure 1.1. Research issues and applications of face processing.

to produce animation. The animation may be used as avatar-based interface
for human computer interaction. One particular application is model-based
face video coding. The idea is to analyze face video and only transmit a few
motion parameters, and maybe some residual. Then the receiver can synthesize
corresponding face appearance based on the motion parameters. This scheme
can achieve better visual quality under very low bit-rate.
In this topic, we present a 3D face processing framework for both analysis
and synthesis. The framework is illustrated in Figure 1.2. Due to the complex-
ity of facial motion, we first collect 3D facial motion data using motion capture
devices. Then subspace learning method is applied to derive a few basis. We
call these basis Geometric Motion Units, or simply MUs. Any facial shapes can
be approximated by a linear combination of the Motion Units. In face motion
analysis, the MU subspace can be used to constrain noisy 2D image motion for
more robust estimation. In face animation, MUs can be used to reconstruct fa-
cial shapes. The MUs, however, are only able to model geometric facial motion
because appearance details are usually missing in motion capture data. These
appearance details caused by motion are important for both human perception
and computer analysis. To handle the motion details, we incorporate appear-