FACE APPEARANCE SYNTHESIS USING FLEXIBLE APPEARANCE MODEL - 3D Face Processing: Modeling, Analysis and Synthesis - page 85

Chapter 8
FACE APPEARANCE SYNTHESIS USING
FLEXIBLE APPEARANCE MODEL
In this chapter, we discuss the face appearance synthesis using flexible ap-
pearance model described in Chapter 6. Our goal is to use appearance model
in a flexible way to synthesize plausible facial appearance variations caused
by lighting and motion. The synthesized appearance can provide visual cues
in synthetic face based interactions. For this purpose, the appearance models
need to be applicable for different lighting conditions and people. Because
we aim at reproducing important visual cues related to facial motion, we ne-
glect less essential phenomena such as specular reflection and assume faces are
Lambertian.
In Section 1, we first describe how to synthesize the illumination effects
of neutral face appearance in different lighting conditions. Next, we describe
works on using the face relighting technique for face recognition under varying
lighting conditions 2. In Section 3 the synthesis of facial appearance caused by
motion is discussed.
1. Neutral Face Relighting
We present a ratio-image based technique [Wen et al., 2003] to use a radiance
environment map to render diffuse objects with different surface reflectance
properties. This method has the advantage that it does not require the separation
of illumination from reflectance, and it is simple to implement and runs at
interactive speed. In order to use this technique for human face relighting, we
have developed a technique that uses spherical harmonics to approximate the
radiance environment map for any given image of a face. Thus we are able
to relight face images when the lighting environment rotates. Another benefit
of the radiance environment map is that we can interactively modify lighting
by changing the coefficients of the spherical harmonics basis. Finally we can
modify the lighting condition of one person's face so that it matches the new
91