PerformingNon-Rigid Structure fromMotion - Deformable Surface 3D Reconstruction from Monocular Images - page 73

6.3.2 LOCAL CONSTRAINTS
As for template-based reconstruction, while global geometric constraints are mostly effective to
reconstruct simple global deformations, local approaches are in general better suited to account for
complex deformations. This is still the case when compared to the quadratic models, which, as
depicted in Fig. 6.11 , produce deformation modes similar to those of the learned linear models of
Section 4.2.2.1 , but without requiring training data. In this section, we present several approaches
to incorporating local smoothness in NRSFM. While some of them are introduced to replace the
linear subspace model, others are used in conjunction with it: Since the shape basis is learned
during reconstruction, which is underconstrained, it can still be improved by imposing additional
smoothness.
The first local smoothness term in NRSFM was introduced in Torresani et al. [ 2001 ]. As in the
original Snakes Kass et al. [ 1988 ], the local constraints are encoded as a regularizer on neighboring
points. More specifically, for neighboring points q i 1 and q i 2 , the regularization term is written as

N s

α i 1 ,i 2 S k,i 2 ) 2 , ( S k,i 1 − (6.20)

k = 1

where S k,i contains the 3D coordinates corresponding to point i in basis S k . The neighborhood can
be established by nearest neighbor search or Delaunay triangulation in image space. The weights
α i 1 ,i 2 are taken to be inversely proportional to the 2D distance between points i 1 and i 2 .
In Olsen and Bartoli [ 2008 ], a similar idea was proposed, but with an additional temporal
component. The regularizer exploits the fact that the tracks of two simultaneously visible points
should have similar shapes. This can be written as
α i 1 ,i 2 q i 2

2

q i 1 − , (6.21)

2

where q i is the vector concatenating the 3D coordinates of point i in the set of frames J . As before,
the weights α i 1 ,i 2 are taken as inversely proportional to the 2D distance between the points. These
constraints are included for all pairs of points that have been tracked simultaneously for at least 10
frames.
Another approach to incorporating local constraints into NRSFM is to assume local rigidity
of the deforming surface. In Llado et al. [ 2010 ], it was assumed that, while some points on the
surface deform, others only move rigidly throughout the sequence. The problem then becomes
one of distinguishing rigid from non-rigid motion, which can be done automatically Llado et al.
[ 2010 ]. As the methods relying on shape repetitions Rabaud and Belongie [ 2008 ], Zhu et al. [ 2010 ]
introduced in Section 6.2 , this involves verifying how well points satisfy epipolar geometry. Since
many points also move non-rigidly, and therefore should be considered as outliers in the fundamental
matrices computation, a RANSAC algorithm is employed. Furthermore, to speedup this procedure,
a degree of non-rigidity score is defined and used to build a prior to guide the RANSAC algorithm.