Graphics Reference
In-Depth Information
fit can be performed in this local neighborhood (7 pixels used in the author's experiments) of
each pixel solving for
φ
c
,
φ
m
, and y:
y
(
ϕ
d
(
m
))
y
))
2
min
φ
c
,φ
m
,
φ
c
(
m
)
−
(
m
φ
m
(
m
)
−
sin (2
.
(1.32)
For large motion, the first-order Taylor degrades, and instead of using the second-order
approximation, a faster solution is to use a simulation that estimates
y
for different values
of
from
an estimated biased
y
. In this case, a median filter is first applied for robustness. Despite
high speed acquisition and motion compensation, imperfections essentially due to sensor
noise, residual uncompensated motion and acquisition conditions as illumination may persist.
To deal with these problems, Ouji et al. (2011) proposed to apply a 3D temporal super-
resolution for each couple of successive 3D point sets
θ
and to create a lookup-table (LUT), which is then used to retrieve the true
θ
M
t
at time
t
. First, a 3D
nonrigid registration is performed. The registration can be modeled as a maximum-likelihood
estimation problem because the deformation between two successive 3D faces is nonrigid in
general. The coherent point drift (CPD) algorithm, proposed in Andriy Myronenko (2006),
is used for the registration the of 3D points set
M
t
−
1
and
M
t
−
1
with the 3D points set
M
t
. The CPD
M
dst
as a probability density
estimation problem and fits the Gaussian Mixture Model (GMM) centroids representing
M
src
and
algorithm considers the alignment of two point sets
M
src
to the data points of
M
dst
by maximizing the likelihood as described in Andriy Myronenko
(2006).
N
src
is the number of points of
m
src
and
M
src
={
sn
|
n
=
1
,...,
N
src
}
.
N
dst
constitutes
the number of points of
M
dst
and
M
dst
={
dn
|
n
=
1
,...,
N
dst
}
. To create the GMM for
M
src
, a multivariate Gaussian is centered on each point in
M
src
. All gaussians share the same
2
I
, I being a 3
2
isotropic covariance matrix
σ
×
3 identity matrix and
σ
the variance in all
directions Andriy Myronenko (2006). Hence the whole point set
M
src
can be considered as a
GMM with the density
p
(
d
) as defined by
N
dst
∝
N
s
m
,σ
2
I
1
N
dst
p
(
d
)
=
p
(
d
|
m
)
,
d
|
m
(1.33)
m
=
1
The core of the CPD method is forcing GMM centroids to move coherently as a group, which
preserves the topological structure of the point sets as described in Andriy Myronenko (2006).
The coherence constraint is imposed by explicit re-parameterization of GMM centroids'
locations for rigid and affine transformations. For smooth nonrigid transformations such as
expression variation, the algorithm imposes the coherence constraint by regularization of the
displacement field Myronenko and Song (2010). Once registered, the 3D points sets
M
t
−
1
and
M
t
and also their corresponding 2D texture images are used as a low resolution data to create
a high resolution 3D point set and its corresponding texture. 2D super-resolution technique as
proposed in Farsiu et al. (2004) is applied, which solves an optimization problem of the form:
minimize
E
data
(
H
)
+
E
regular
(
H
)
.
(1.34)
The first term
E
data
(
H
) measures agreement of the reconstruction
H
with the aligned low
resolution data.
E
regular
(
H
) is a regularization or prior energy term that guides the optimizer
