Introduction - Deformable Surface 3D Reconstruction from Monocular Images - page 1

CHAPTER 1

Introduction

Deformable surface 3D reconstruction from monocular images is an active area of research in the
Computer Vision community. This encompasses recovering both the shape of thin objects that can
be treated as surfaces without perceptible thickness and the visible envelope of fully 3D objects.
Whereas this may seem easy for a human being, it remains a challenging and ambiguous problem
for computer-based techniques.This is especially true when the sensor data is noisy, which is typically
the case when dealing with real images.
Apart from being a fascinating problem, non-rigid 3D shape recovery has applications in
many different domains:

￿ The entertainment industry could benefit greatly from improved techniques for video-based
shape recovery. In animation movies, video games, or special effects, many things are still done
manually, image after image. As illustrated by Fig. 1.1 , there are already effective techniques
for handling 2D surface deformations in an Augmented Reality context. However, they must
be extended to full 3D deformations to better account for phenomena such as self-occlusion
and self-collisions that become prevalent as the deformations become larger. This could be
used to draw virtual advertisement logos on athletes' or fashion models' clothes, thus avoiding
the need to physically print them and making it easy to change them as necessity dictates.
Similarly, a lot of time could be saved if the deformations of the clothes of animated characters,
such as those of Fig. 1.2 , could simply be obtained by filming a real person performing some
motion, reconstructing his or her clothes in 3D, and re-applying the resulting deformations
to the animated character.

￿ Many sports could benefit from a system that reconstructs non-rigid 3D shapes from video. For
example, as shown in Fig. 1.3 (a,b), sailors want to analyze the effect of their maneuvers on the
shape of their sails, or, sometimes even more interestingly, study the sails of their opponents.
In this context, video presents a clear advantage over other sensors that should be placed on
the sail itself, thus changing its behavior. Similarly, analyzing the deformations of any sports
structure, such as the skis or the plane wings of Fig. 1.3 (c,d), in realistic situations could help
improving their design.

￿ More speculatively, in the medical field, the current trend is to make surgery ever less invasive.
This implies smaller and smaller cuts in the patient's skin, which do not give the surgeons a
direct view of their work. They only leave enough space for small cameras to be introduced
into the patient's body. In such conditions, the resulting images are of poor quality, and make