Image Processing Reference

In-Depth Information

CHAPTER

6

Performing Non-Rigid

Structure fromMotion

In the previous chapter, we have shown that recovering non-rigid structure and motion from
N
c

points tracked in
N
f
frames could theoretically be done by factorizing a measurement matrix into

a product of two matrices. This can be expressed as

W

=

CB
,

(6.1)

where
W
is the measurement matrix,
C
contains products of the shape coefficients with the motion

parameters, and
B
contains the shape basis. However, as mentioned in Section
5.4
, this factorization is

subject to ambiguities. The decomposition can only be computed up to an invertible transformation

G
, up to a global scale, and up to ambiguities between shape coefficient values and basis shape

magnitudes. In addition to those theoretical ambiguities, the problem also is ill-conditioned due to

the presence of image noise. As a consequence, constraints must be incorporated into the factorization

to overcome these issues.

In general, adding constraints to the factorization of Eq.
6.1
, yields an optimization problem

that can be parameterized in two different ways. The first one involves expressing the reconstruction

in terms of the corrective transform
G
only. This implicitly satisfies the measurement constraints,

since
W

B
, where
C
and
B
are the matrices obtained by SVD. Therefore, only

the additional regularization terms are taken into account to find the best
G
. The second way is to

write the objective in terms of the original variables
S
k
,
c
k
, and
R
j

C B

CGG
−
1

=

=

of Eqs.
5.6
or
5.12
, as well as
A

and
t
j

if also optimized. This yields an optimization problem of the form

C
(c
k
,
R
j
)
B
(
S
k
)

2

−

minimize

S
k
,c
k
,
R
j

W

,

(6.2)

F

where
C
and
B
are expressed as functions of the variables, and

·
F
is the Frobenius norm. Addi-

tional knowledge can then be introduced either as hard constraints or as regularizers in the objective

function.

In this chapter, we review the different kinds of additional constraints that have been proposed

in recent years. As in the case of template-based reconstruction, temporal and geometric consistency

constraints have been used. In addition to these, an NRSFM-specific constraint arises from the fact

that the estimated rotation matrices must be orthonormal. We will start with this one and then move

on to temporal and geometric ones. Note that we do not differentiate between the weak and full

perspective cases, since these constraints generally apply to both.

Search WWH ::

Custom Search