Image Processing Reference
Figure 6.1: Ambiguities of orthonormality constraints Akhter et al. [ 2009 ]. (Left) Reconstruction of
a face. The cost function suggests that many shapes satisfy orthonormality constraints. However, the
shapes in this region are all the same up to a Euclidean transformation. (Right) Reconstruction of a cube
from noise-free data. While the method of Bregler et al. [ 2000 ] does not give a correct solution (a), using
orthonormality constraints gives a perfect reconstruction (b). As before, there exist several solutions, but
they are all the same up to a global rotation (d-f ). Courtesy of I. Akhter. © 2009 IEEE.
in Torresani et al. [ 2001 ], the authors relied on an iterative scheme that involved alternatively opti-
mizing rotations, shape basis, and shape coefficients. Orthonormality constraints were implicitly sat-
isfied by parameterizing the rotations with exponential coordinates. For similar reasons, in Llado et al.
[ 2010 ], the rotations were parameterized with quaternions. By contrast, in Brand [ 2005 ] it was pro-
posed to directly minimize the squared error induced by the orthonormality constraints of Eq. 6.3 .
A variable-metric quasi-Newton scheme was used and the constraint
1 was added. In
a similar constrained optimization paradigm, an algorithm that enforced orthonormality by con-
straining the solution to remain on a Riemannian manifold was developed in Shaji and Chandran
[ 2008 ]. Fig. 6.2 depicts the results obtained with this method on the shark sequence of Torresani et al.
[ 2008 ], which is used in many publications.
Despite the fact that orthonormality constraints strongly reduce the ambiguities of NRSFM,
resulting solutions often remain sensitive to image noise. As a consequence, additional constraints
need to be introduced. These constraints can be roughly classified into temporal consistency and
geometric constraints, both of which are discussed below.
G k F
As shown in Chapter 4 for template-based reconstruction, accounting for the fact that, in a video
sequence, the shape does not vary arbitrarily from frame to frame gives very strong reconstruction
cues. This kind of knowledge is even better adapted to NRSFM, since it is specifically designed
to deal with sequences as opposed to single frames. As a consequence, various types of temporal
constraints have been proposed to improve structure and motion recovery.