Image Processing Reference
In-Depth Information
CHAPTER
6
Performing Non-Rigid
Structure fromMotion
In the previous chapter, we have shown that recovering non-rigid structure and motion from
N
c
points tracked in
N
f
frames could theoretically be done by factorizing a measurement matrix into
a product of two matrices. This can be expressed as
W
=
CB
,
(6.1)
where
W
is the measurement matrix,
C
contains products of the shape coefficients with the motion
parameters, and
B
contains the shape basis. However, as mentioned in Section
5.4
, this factorization is
subject to ambiguities. The decomposition can only be computed up to an invertible transformation
G
, up to a global scale, and up to ambiguities between shape coefficient values and basis shape
magnitudes. In addition to those theoretical ambiguities, the problem also is ill-conditioned due to
the presence of image noise. As a consequence, constraints must be incorporated into the factorization
to overcome these issues.
In general, adding constraints to the factorization of Eq.
6.1
, yields an optimization problem
that can be parameterized in two different ways. The first one involves expressing the reconstruction
in terms of the corrective transform
G
only. This implicitly satisfies the measurement constraints,
since
W
B
, where
C
and
B
are the matrices obtained by SVD. Therefore, only
the additional regularization terms are taken into account to find the best
G
. The second way is to
write the objective in terms of the original variables
S
k
,
c
k
, and
R
j
C B
CGG
−
1
=
=
of Eqs.
5.6
or
5.12
, as well as
A
and
t
j
if also optimized. This yields an optimization problem of the form
C
(c
k
,
R
j
)
B
(
S
k
)
2
−
minimize
S
k
,c
k
,
R
j
W
,
(6.2)
F
where
C
and
B
are expressed as functions of the variables, and
·
F
is the Frobenius norm. Addi-
tional knowledge can then be introduced either as hard constraints or as regularizers in the objective
function.
In this chapter, we review the different kinds of additional constraints that have been proposed
in recent years. As in the case of template-based reconstruction, temporal and geometric consistency
constraints have been used. In addition to these, an NRSFM-specific constraint arises from the fact
that the estimated rotation matrices must be orthonormal. We will start with this one and then move
on to temporal and geometric ones. Note that we do not differentiate between the weak and full
perspective cases, since these constraints generally apply to both.
Search WWH ::
Custom Search