Future Directions - Deformable Surface 3D Reconstruction from Monocular Images - page 80

CHAPTER 7

Future Directions

In this survey, we have reviewed several template-based and non-rigid structure-from-motion tech-
niques that can be used to robustly recover 3D shape given point correspondences. The former can
be made very reliable when a template is available but are of course inappropriate otherwise, which
is often the case in practice. When video sequences are available, the latter can be invoked instead
and are very effective when the deformations are not too complex.
In both cases, shape recovery implies solving an ill-posed problem and additional geometric
or temporal consistency constraints are needed for good results. Furthermore, when there are too few
correspondences, for example because the surfaces are relatively featureless, neither class of techniques
performs well, which greatly limits their applicability. To remedy this situation, we believe that future
research should focus on taking advantage of additional sources of image information, such as

￿ Silhouettes: The projected contours of a surface give powerful clues as to their 3D shape. They
already have been extensively exploited to reconstruct developable surfaces Gumerov et al.
[ 2004 ], Perriollat and Bartoli [ 2007 ], as discussed in Chapter 4 . However, these approaches
do not naturally generalize to non-developable surfaces whose shape cannot be inferred from
their outlines, which often are occluding contours. Such contours have been used for 3D surface
reconstruction Ilic et al. [ 2007 ], Sullivan et al. [ 1994 ], Szeliski and Weiss [ 1998 ] but most
existing approaches rely on iterative schemes in which the occluding contours are predicted
from a current shape estimate and compared to their observed image locations. This runs
contrary to the spirit of the most effective template-based method that perform reconstruction
either in closed form or by finding the minimum of a convex function. Further work is therefore
required to merge these two different strands of research.

￿ Texture: Inferring shape from correspondences requires texture, since the correspondences
typically only are established between interest points. However, this only uses a fraction of
the available information; The orientation of the patches surrounding the interest points
can also be inferred from textural deformations Hinterstoisser et al. [ 2011 ]. In other words,
when correspondences can be established, it is usually also possible to estimate the surface
normals. In Moreno-Noguer et al. [ 2009 ], such estimates were used to relax the inextensibility
constraints required by an earlier template-based method Salzmann et al. [ 2008a ], while still
computing the 3D shape in closed-form. Other approaches discussed in this survey could and
should be similarly extended to make the most of the texture, especially when the surface is
not uniformly well-textured.