Image Processing Reference

In-Depth Information

which is resolved by using shading information and, when necessary, smoothness constraints. This

approach uses more of the textural information than all those that rely solely on interest points,

which ignore most of the image pixels. As discussed in Chapter
2
, when operating in a well-defined

domain such as face reconstruction for which there exists not only a geometric model but also an

appearance model, it becomes possible to use the image texture even more extensively by using an

analysis-by-synthesis approach
Blanz and Vetter
[
1999
],
Romdhani and Vetter
[
2003
] to estimate

both shape and illumination parameters, as shown in Fig.
4.10
.

While the mesh-based parameterization is the most common representation for template-

based reconstruction, it is not the only possible one. As mentioned in Section
2.3
, control points

based parameterizations have been proposed to model non-rigid objects. Recently, in
Brunet
et al.

[
2010
], a free-form deformation model was recently used for monocular reconstruction. This model

has the advantage of making it easy to compute a global smoothness regularizer by exploiting the

second derivatives of the B-spline basis functions that define the deformations. This regularizer used

in conjunction with additional distance constraints was shown to outperform several state-of-the-art

methods in terms of reconstruction accuracy.

4.2.2.2 Local Smoothness

The methods of Section
4.2.2.1
, which rely on regularization models expressed as linear combinations

of deformation modes, are good at recovering the shape of surfaces that deform relatively smoothly.

However, they do not perform as well when deformations are more local or sharper, such as those

depicted by Figs.
4.4
and
4.11
where folds appear on the surface. In theory, handling such local

deformations could be achieved by using a much larger number of global modes. However, in practice,

this would mean introducing far more variables—the weights associated to the modes—which, for

computational reasons, could easily make the previous approaches impractical.

An approach to overcoming this problem by replacing global smoothness constraints with

local ones was introduced in
Salzmann
et al.
[
2008b
]. It starts from the following observations. First,

locally, all parts of a physically homogeneous surface obey the same deformation rules. Second, these

local deformations are more constrained than those of the global surface and can be learned from

fewer examples. To exploit this, it is the manifold of local, as opposed to global, surface deformations

that is represented. In
Salzmann
et al.
[
2008b
], these local models were learned using a nonlinear

technique. However, this yields non-convex objective functions, and is therefore only appropriate in

a tracking framework. Thus, these nonlinear models were later replaced by linear ones, where each

local patch is represented as a linear combination of modes
Salzmann and Fua
[
2011
].

In
Salzmann and Fua
[
2011
], this representation was used to regularize the reconstruction of

the global surface by penalizing large local shape deviations from the learned linear manifold. To

this end, as shown in Fig.
4.12
, the mesh representing the surface is subdivided into overlapping

patches. Each patch is taken as an
N
p
×
N
p
square mesh, with
N
p
=

5 in
Salzmann and Fua
[
2011
].

Note that this does not truly limit the approach to rectangular surfaces, since patches can be defined

partially outside the global shape. Given an instance of a surface, each one of its local patches is

Search WWH ::

Custom Search