FACE APPEARANCE SYNTHESIS USING FLEXIBLE APPEARANCE MODEL - 3D Face Processing: Modeling, Analysis and Synthesis - page 89

If we are given photographs of two people's faces under different lighting
conditions, we can modify the first photograph so that it matches the lighting
condition of the second face. We first compute the radiance environment map for
each face. If the two faces have the same average albedo, then the two radiance
environment maps have the same albedo, and we can apply equation (8.11) to
relight the first face to match the second face's lighting condition. In practice,
if two people have similar skin colors, we can apply equation (8.11) to relight
one face to match the lighting condition of the other.
Given one input face photograph and a user interface to edit the 9 coefficients
of the radiance environment map, we can interactively synthesize novel illumi-
nation effects on the face image based on technique described in Section 1.1.4.
1.2.1 Dynamic range of images
Because digitized image has limited dynamic range, ratio-based relighting
would have artifacts where skin pixel values are too low or nearly saturated.
To alleviate the problem, we apply constrained texture synthesis for these pix-
els. Our assumption is that the high frequency face albeo, similar to texture,
contains repetitive patterns. Thus we can infer local face appearance at the
places of artifacts from examples on other part of the face. We first identify
these pixels as outliers that do not satisfy Lambertian model using robust statis-
tics [Hampel and et. al, 1986]. Then we use the remaining pixels as example to
synthesize texture at the place of outliers. We use a patch-based Image Analogy
algorithm [Hertzmann and et. al., 2001], with the constraint that a candidate
patch should match the original patch up to a relighting scale factor. Since we
use a patch-based approach and we only apply it to the detected outlier regions,
the computation overhead is very small. Figure 8.1 shows an example where
Figure 8.1. Using constrained texture synthesis to reduce artifacts in the low dynamic range
regions. (a): input image; (b): blue channel of (a) with very low dynamic range; (c): relighting
without synthesis; and (d): relighting with constrained texture synthesis.
the low dynamic range of the image causes artifacts in the relighting. (a) is the