Future Directions - Deformable Surface 3D Reconstruction from Monocular Images - page 81

￿ Shading: It has long been known as a useful but fragile source of shape informa-
tion Horn and Brooks [ 1989 ], which naturally complements textural cues where the albedos
are constant or vary slowly Fua and Leclerc [ 1995 ]. As discussed in Chapter 4 , it was used
in White and Forsyth [ 2006 ] to disambiguate the direction of normals obtained from textural
clues, in Moreno-Noguer et al. [ 2009 ] to provide normal estimates around interest points, and
in Moreno-Noguer et al. [ 2010 ] to choose among competing shapes that all result in roughly
the same image projections. The shading models used by these algorithms, however, remain
simplistic. They would need to be extended further to prove truly useful outside of very special-
ized applications, such as virtually flattening a book to produce better photocopies Zhang et al.
[ 2004 ]. We believe that a promising direction is to use modern statistical learning techniques
to relate gray level patterns within image patches to local 3D shape estimates using realistic
training data and without making unwarranted assumptions.
In addition to means of exploiting the image data more thoroughly, better and more widely
applicable deformation models are required to break the ambiguities that plague monocular 3D sur-
face reconstruction. When the surface is made of a material that is known a priori , effective models
can be learned offline using training data. In the more general case when the surface material is not
known beforehand, the model could be learned online using the parts of the surface that are suffi-
ciently well-textured for a very simple regularizing prior to be enough to obtain valid reconstructions.
This model could then be used to constrain the reconstruction of the rest of the surface. For similar
purposes, one could also exploit transfer learning techniques that leverage labeled data of related
problems to learn a model for a different problem where no, or very few, labeled data is available. In
our context, given the training examples for some materials, we could learn a deformation model for
a new material from very small amounts of reconstructed 3D shapes.
In short, current monocular approaches to 3D surface reconstruction can be well formalized
and already yield promising results on well-textured surfaces. Much work is still required to make
them fully operational on less well-textured surfaces but the way forward seems relatively clear.