Modeling and Synthesis of Realistic Visual Speech in 3D - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 275

points rather easy. At that point all 116 points are in good correspondence.
Further snapshots of the example face are no longer handled manually. From the
initial frame, the points are tracked automatically throughout the video. The
tracker looks for point candidates in a neighborhood around their previous
position. A dark blob is looked for and its midpoint is taken. As data are sampled
at video rate, the motions between frames are small and this very simple tracking
procedure only required manual help at a dozen or so frames for the set of
example data. The main reason was two candidate points falling into the search
region. Using this tracker, correspondences for all points and for all frames could
be established with limited manual input.
In order to find the deformations for the visemes, the corresponding frames were
selected from the video and their 3D reconstructions were made. The 3D
positions of the 116 points served as anchor points, to map all vertices of the
generic model to the data. The result is a model with the shape and expression
of the example face and with 2'268 vertices at their correct positions. This
mapping was achieved with the help of Radial Basis Functions.
Radial Basis Functions (RBFs) have become quite popular for face model fitting
(Pighin et al., 1998; Noh et al., 2001). They offer an effective method to
interpolate between a network of known correspondences. RBFs describe the
influence that each of the 116 known (anchor) correspondences have on the
nearby points in between in this interpolation process.
Consider the following equations,

n

=+ ∑ y y ω d

(1)

i i j j new

j = 1

which specify how the positions y i of the intermediate points are changed into

i y under the influence of the n vertices m j of the known network (the 116 vertices
in our case). The shift is determined by the weights

new

ω j and the virtual displacements
d j that are attributed to the vertices of the known network of correspondences. More
about these displacements is to follow. The weights depend on the distance of the
intermediate point to the known vertices:

hs r

ω = (/) s = ym

(2)

j j j i j

r , where r is a cut-off value for the distance beyond which h is put to zero,
and where in the interval [0, r ] the function h ( x ) is of one of two types:

For s j ≤