CONCLUSION AND FUTURE WORK - 3D Face Processing: Modeling, Analysis and Synthesis - page 110

is based on the geometric tracking results. In other words, a closer coupling of
the geometry and appearance processing improves the overall performance.
To improve our face processing, we plan to investigate closer correlations
between geometry and appearance, following the direction mentioned in [Cascia
et al., 2000] and [Vacchetti et al., 2003]. In our current geometric tracking
algorithm, template-matching based optical-flow is used to estimate the frame
to frame motions. To alleviate the error accumulation (i.e. drifting) problem
of long term template-matching, it uses the templates from the previous frames
and the first frames. For example, the even nodes of the face mesh is tracked
using templates from the previous frame and the odd nodes are tracked using
templates from the first frame. However, only the templates in the first frame
and the previous frame do not give a sufficient description of the appearance
variations of the templates. Therefore, the tracking is not accurate when the out-
of-plane rotation angle is large (e.g. larger than 40 degrees). In the future, we
plan to adopt better statistical models of the templates by (1) deriving statistical
models offline using rendering of 3D face databases; (2) adopting the novel
appearance features proposed in Chapter 7 that are less illumination and person
dependent; and (3) online adaptation of the models using approaches suggested
by [Jepson et al., 2001]. Then these models of template appearance can be used
to improve the robustness of geometric tracking under large 3D pose variations,
partial occlusions and etc.
2.3 Human perception evaluation of synthesis
Human perception evaluation of face synthesis should be done in the context
of specific applications such as lip reading. Hypotheses about visual factors
need be created and tested in the evaluation. These hypotheses can then be used
to improve face synthesis so that the synthetic animation can be more effective
for applications.
23.1 Previous work
To evaluate the quality of synthetic face animation, one approach is to com-
pare the synthetic face with the original face in terms of reconstructed error.
This approach is used by (1) low bit-rate coding oriented face animation such as
MPEG-4 FAPs [Eisert et al., 2000, Tu et al., 2003]; and (2) machine learning-
based data-driven face animation such as [Brand, 1999, Guenter et al., 1998].
However, the synthetic face motion can be different from the real motion
but still looks natural. Consequently, human subject evaluations are important.
Human evaluation with a small set of subjects were used in [Brand, 1999, Ezzat
et al., 2002]. On the other hand, in many scenarios mimicking real human face
motion is not the only goal for synthetic face animation. For example, non
photo-realistic styles and abstraction can be created to convey certain informa-
tion in face animation [Buck and et al., 2000, Chen et al., 2002]. Another type