FormalizingTemplate-Based Reconstruction - Deformable Surface 3D Reconstruction from Monocular Images - page 17

CHAPTER 3

FormalizingTemplate-Based
Reconstruction

In this chapter, we focus on template-based approaches to monocular 3D reconstruction and intro-
duce the general formulation of this problem that is common to most such methods. To this end,
we rely on a triangulated surface representation and two different kinds of camera models, which
we introduce first. We then discuss the 3D-to-2D correspondences that serve as input and derive a
linear problem formulation. It is undersconstrained, but forms the basis of many of the techniques
of Chapter 4 that impose different kinds of constraints to resolve the ambiguities.

3.1 PROBLEMDEFINITION
Template-based non-rigid 3D reconstruction can be defined as the problem of inferring the 3D
shape of a surface in an input image, given a reference image in which the 3D surface shape is
known. Although other surface parameterizations are possible, triangulated meshes are the most
common in these kinds of approaches. We will therefore assume that the 3D shape is represented
as a triangulated mesh with N v vertices and N t facets. The goal then is to recover the 3D vertex
locations such that the shape best corresponds to what is observed in the input image.

3.1.1 MOTIVATION
To derive the formulation below, we assume that we can establish correspondences such as those
depicted by Fig. 3.1 between the reference and input images. Two main reasons motivated this
choice:

￿ Establishing correspondences between two images does not involve strong assumptions, apart
from requiring the surface to be textured. Furthermore, given a reference image, correspon-
dences can be established using either a single input image or a whole video sequence, which
means we do not have to track points from image to image, but can if we want to. Conse-
quently, the insights presented here can be used to understand the behavior both of algorithms
that rely on correspondences between model and input images, such as those discussed in
Chapter 4 , and of structure-from-motion algorithms, such as those introduced in Chapter 5
and Chapter 6 .

￿ As shown in Salzmann et al. [ 2007b ], relying on image correspondences makes it possible to
formalize the shape recovery problem as one of solving an ill-conditioned linear system and