Image Processing Reference
Figure 6.2: Reconstruction of the shark data using the method of Shaji and Chandran [ 2008 ]. Here,
orthonormality is enforced by constraining the solution to remain on a Riemannian manifold. Red
circles correspond to ground-truth points, and blue dots to reconstructed ones. Courtesy of S. Chandran.
© 2009 IEEE.
In Section 4.1 , we showed that zeroth order motion models are effective to constrain template-
based frame-to-frame reconstruction. This remains true in NRSFM, where they have been used
extensively Aanaes and Kahl [ 2002 ], Del Bue et al. [ 2007 ], Rabaud and Belongie [ 2008 ]. The in-
tuition behind such models simply is that the variation of the shape Q
= k c k S k
consecutive frames is small. Therefore, the term
j = 2
Q j − 1
i = 1
can be added to the objective function of Eq. 6.2 . Typically, the weight λ s , which accounts for
the relative influence of the two terms, is set manually. In Torresani et al. [ 2008 ], a similar, though
more general, linear dynamical model was introduced in a probabilistic framework. In that case, the
temporal structure takes the form
c j − 1
where c j is the vector of all shape coefficients for frame j , is an N s × N s transition matrix, and
η j is a zero-mean Gaussian noise vector. This model represents the shape coefficients in a frame as
a linear function of those in the previous one. If is taken to be the identity matrix, this essentially
becomes equivalent to the previous zeroth order motion model.
For the same reasons that make it permissible to penalize large frame-to-frame shape
variations, it can be assumed that the camera motion between two consecutive frames is small.
In Rabaud and Belongie [ 2008 , 2009 ], this was done by relying on the same zeroth order motion