Image Processing Reference

In-Depth Information

We will first introduce a general formulation of NRSFM under a weak perspective, or affine,

camera model. In this case, the projection of a 3D point
q
i

can be written as

d
u
i

v
i

=

Rq
i
+

t
,

(5.1)

where
R
contains the first two rows of the full camera rotation matrix, and
t
is the 2

1 camera

translation vector. Assuming no distortion, the matrix of internal camera parameters reduces to a

single focal length, which was here absorbed by the scalar
d
. Since, in NRSFM, the notion of facet

is absent, the same
d
is used for all the points. We will then discuss NRSFM under full perspective

projection, where the projection of a 3D point
q
i

×

is expressed as

⎡

⎤

u
i

v
i

1

A
Rq
i
+

t
,

⎣

⎦
=

d
i

(5.2)

with
A
the 3
×
3 matrix of internal camera parameters.

As in the case of template-based reconstruction, correspondences alone are insufficient for

unambiguous reconstruction. To reduce the ambiguities, most NRSFM methods rely on a linear

subspace model to constrain the deformations of the 3D points. Whereas in the template-based

case, the deformation modes could be infered from the reference mesh using a technique such

as
Salzmann
et al.
[
2007c
], this is no longer the case for NRSFM. Consequently, the modes are

typically taken as additional unknown variables and recovered at the same time as their coefficients,

along with the rotation and translation for each frame of the sequence. Note that, since the modes

are obtained neither from a large dataset of deformed surfaces as in
Salzmann
et al.
[
2007c
] nor

by modal analysis, they do not necessarily favor smooth deformations. Depending on the number

of basis shapes involved, they rather encourage the deformations to remain simple. Since most

NRSFM techniques exploit this linear subspace representation, we will describe it as part of the

general approach. However, as will be discussed in Chapter
6
, some very recent methods depart

significantly from this initial formulation.

In theory, NRSFM is more generally applicable than template-based shape recovery, since

it requires neither a calibrated camera, nor a reference template. In practice, however, because of

the higher number of degrees-of-freedom, NRSFM methods are subject to more ambiguities and

are more sensitive to measurement noise. As a consequence, to be as effective as template-based

approaches, they often require stronger constraints and a good initialization. In many cases, the

latter is obtained by applying a rigid structure-from-motion algorithm.

In the remainder of this chapter, we start by reviewing the problem formulation for the weak

perspective case as introduced in
Bregler
et al.
[
2000
]. We then discuss NRSFM in the perspective

case, as proposed by
Hartley and Vidal
[
2008
],
Xiao and Kanade
[
2005
].

Search WWH ::

Custom Search