Image Processing Reference

In-Depth Information

6.3.2 LOCAL CONSTRAINTS

As for template-based reconstruction, while global geometric constraints are mostly effective to

reconstruct simple global deformations, local approaches are in general better suited to account for

complex deformations. This is still the case when compared to the quadratic models, which, as

depicted in Fig.
6.11
, produce deformation modes similar to those of the learned linear models of

Section
4.2.2.1
, but without requiring training data. In this section, we present several approaches

to incorporating local smoothness in NRSFM. While some of them are introduced to replace the

linear subspace model, others are used in conjunction with it: Since the shape basis is learned

during reconstruction, which is underconstrained, it can still be improved by imposing additional

smoothness.

The first local smoothness term in NRSFM was introduced in
Torresani
et al.
[
2001
]. As in the

original Snakes
Kass
et al.
[
1988
], the local constraints are encoded as a regularizer on neighboring

points. More specifically, for neighboring points
q
i
1

and
q
i
2
, the regularization term is written as

N
s

α
i
1
,i
2

S
k,i
2
)
2
,

(
S
k,i
1
−

(6.20)

k
=
1

where
S
k,i
contains the 3D coordinates corresponding to point
i
in basis
S
k
. The neighborhood can

be established by nearest neighbor search or Delaunay triangulation in image space. The weights

α
i
1
,i
2
are taken to be inversely proportional to the 2D distance between points
i
1
and
i
2
.

In
Olsen and Bartoli
[
2008
], a similar idea was proposed, but with an additional temporal

component. The regularizer exploits the fact that the tracks of two simultaneously visible points

should have similar shapes. This can be written as

α
i
1
,i
2

q
i
2

2

q
i
1
−

,

(6.21)

2

where
q
i
is the vector concatenating the 3D coordinates of point
i
in the set of frames
J
. As before,

the weights
α
i
1
,i
2
are taken as inversely proportional to the 2D distance between the points. These

constraints are included for all pairs of points that have been tracked simultaneously for at least 10

frames.

Another approach to incorporating local constraints into NRSFM is to assume local rigidity

of the deforming surface. In
Llado
et al.
[
2010
], it was assumed that, while some points on the

surface deform, others only move rigidly throughout the sequence. The problem then becomes

one of distinguishing rigid from non-rigid motion, which can be done automatically
Llado
et al.

[
2010
]. As the methods relying on shape repetitions
Rabaud and Belongie
[
2008
],
Zhu
et al.
[
2010
]

introduced in Section
6.2
, this involves verifying how well points satisfy epipolar geometry. Since

many points also move non-rigidly, and therefore should be considered as outliers in the fundamental

matrices computation, a RANSAC algorithm is employed. Furthermore, to speedup this procedure,

a degree of non-rigidity score is defined and used to build a prior to guide the RANSAC algorithm.

Search WWH ::

Custom Search