FormalizingTemplate-Based Reconstruction - Deformable Surface 3D Reconstruction from Monocular Images - page 20

Figure 3.2: Obtaining image correspondences. A feature point is detected in the reference image, shown
in the middle. Knowing the reference 3D shape of the mesh, on the left, and the camera projection matrix,
we can retrieve the facet to which the feature point belongs, and define the point in terms of its barycentric
coordinates. The feature point can then be matched against points detected in the input image, shown
on the right. This yields 3D-to-2D correspondences in terms of the unknown 3D mesh vertices in the
input image.

that methods addressing the 2D case deserve their own separate review. Therefore, we will limit the
study in this survey to 3D reconstruction techniques.
For template-based approaches, the matches are established between the current image of
interest and the reference image, in which the 3D shape and the camera calibration are known, as
depicted in Fig. 3.2 . Under such assumptions, the 3D locations of the feature points on the template
can be computed by intersecting the ray between the camera center and the 2D image measurement
with the facets of the triangulated mesh. This lets us represent a 3D point in terms of its barycentric
coordinates with respect to the vertices of the facet intersected by the ray. This yields 3D-to-2D
correspondences for the current image, where the 3D positions of the feature points are defined
with respect to the unknown 3D positions of the mesh vertices. To recover the 3D shape, the idea is
then to find the position of the mesh vertices that minimizes the distance between the detected 2D
features and the 3D points locations projected into the image.

3.3 LINEAR FORMULATION

In this section, we show that recovering the 3D shape of a non-rigid surface from 3D-to-2D
correspondences such as those introduced in Section 3.2 amounts to solving a linear system. Under
the weak perspective projection model, exactly one third of the singular values of the corresponding
matrix are zero, which accounts for depth ambiguities. Under the full perspective model, only one
is strictly zero but the same one third are so small as to make the system extremely ill-conditioned.

3.3.1 AMBIGUITIES UNDERWEAK PERSPECTIVE PROJECTION
We now show how computing the 3D mesh vertex coordinates given 3D-to-2D correspondences
in the weak perspective case can be formulated as the solution to a linear system and discuss its