Image Processing Reference

In-Depth Information

strong constraints as they did in the template-based case, where they were pre-computed and fixed.

This explains why additional smoothness constraints are often necessary in NRSFM.

One of the first global geometric constraints employed in NRSFM involved assuming that

the mean shape is the dominant component of the shape in each frame. Constraining the surface

reconstruction can then be done by encouraging the shape in each frame to remain close to the

unknown mean shape
Brand
[
2001
], or close to an initial estimate computed using rigid structure

from motion techniques
Aanaes and Kahl
[
2002
]. In essence, this assumption simply means that

the object mostly moves rigidly, and is thus only valid for small deformations.

As mentioned in Section
5.4
, recovering the basis shapes is an ambiguous problem, since

there may be dependencies between them. As a consequence, there is a global affine ambiguity

of the shape basis. Therefore, other types of geometric constraints were proposed to disambiguate

the computation of the shape basis. The methods of
Xiao and Kanade
[
2005
],
Xiao
et al.
[
2004b
]

established basis constraints in a similar manner as the orthonormality constraints of Eqs.
6.4
and
6.5
.

To this end, they rely on the condition number of sub-matrices of
W
to find the most independent
N
s

images in the sequence. The corresponding, unknown 3D shapes are then taken as the basis shapes.

Therefore, constraints arise from the fact that the surface in the chosen frames must be generated

by a single basis shape. Ordering the frames so that the chosen ones are the first
N
s
frames in the

sequence, this yields constraints of the form

c
i

=

1
,
1

≤
i
≤
N
s
,

(6.14)

c
i

=

0
,
1

≤
i, j
≤
N
s
,i
=
j.

Following the same approach as for the orthonormality constraints of Eqs.
6.4
and
6.5
, these con-

straints can be used to derive linear equations in terms of the quadratic corrective transform
H
k
.

When used in conjunction with the orthonormality property, these constraints were shown to be

sufficient to remove the ambiguity in NRSFM
Xiao
et al.
[
2004b
]. In Fig.
6.6
, we compare the

results obtained with orthonormality constraints and basis constraints under a weak perspective

model
Xiao
et al.
[
2004b
] and a full perspective one
Xiao and Kanade
[
2005
]. Note that the weak

perspective reconstruction is distorted.

A similar idea as in
Xiao
et al.
[
2004b
] was proposed in
Zhu
et al.
[
2010
]. The shape in a

particular image is assumed to be generated by only a subset of all the basis shapes, and, therefore,

the coefficients vectors
c
j

should be sparse. This is enforced by adding the penalty term

N
f

j
=
1

c
j

λ
c

1
,

(6.15)

c
j

2

to the objective function, while imposing

2
=

1

∀

j
. This proved effective to remove the rotation

ambiguity of the shape basis.

Two other approaches have also been proposed to more directly encourage the basis shapes to

remain independent. In
Bartoli
et al.
[
2008
], a coarse-to-fine approach to recovering the modes was

Search WWH ::

Custom Search