Image Processing Reference

In-Depth Information

temporal smoothness. Similarly, there also is an ambiguity between the magnitude of the basis shapes

and their corresponding coefficients.

Furthermore, the same global rotation-translation ambiguity as in the rigid case re-

mains in non-rigid structure from motion solutions. In the full perspective case, it was shown

in
Hartley and Vidal
[
2008
] that, in the uncalibrated case, the solution is only determined up to

a linear transformation. However, with a calibrated camera, this ambiguity reduces to the same

undetermined global rotation as in the weak perspective case.

Finally, in addition to the ambiguities inherent to the problem that have been formally proved

in the above-mentioned papers, other ambiguities have been observed in practice. In
Torresani
et al.

[
2003
], it was noted that if too many basis shapes were required, the reconstruction problem became

ambiguous. Similarly, in
Bartoli
et al.
[
2008
], the problem that treating all modes equally results

in ambiguities due to the potential dependencies of the modes. Solutions to these problem involv-

ing higher order deformation models, or coarse-to-fine modes computation will be discussed in

Chapter
6
.

5.5 THEMISSINGDATA PROBLEM

A weakness of non-rigid structure-from-motion techniques is their sensitivity to missing data and

mismatches. Several solutions to these problems have been proposed.

The first publications to address the missing data problem were
Brand
[
2001
],
Torresani
et al.

[
2001
]. Both proposed to exploit fully tracked points to infer the missing data. In particular, they

follow the idea introduced by
Irani
[
1999
] for optical flow estimation, and establish a basis flow

using the fully tracked points. More specifically, assuming that
W
has rank
r
, all columns of
W
can

be modeled as linear combinations of
r
basis tracks. If at least
r
points have been tracked throughout

the whole sequence, this basis can be obtained by singular value decomposition of the corresponding

part of
W
. To fill in the missing data in matrix
W
, they follow the formulation of
Lucas and Kanade

[
1981
], and express the locations of the image points in terms of the basis coefficients. The resulting

system of equations is then solved iteratively to obtain the correct coefficients and thus predict the

location of points that were lost during tracking.

While this is effective when some points have been tracked in all images, it would still fail in the

more realistic case where points are visible for some frames and then disappear. In
Olsen and Bartoli

[
2008
], this problem was addressed by separating the sequence into blocks of frames whose motion

parameters can then be estimated. Given the motion in the individual blocks, the basis shapes can

be estimated later by exploiting priors.

A different line of works followed the idea introduced in
Marques and Costeira
[
2009
] to deal

with missing data in rigid structure-from-motion. In this paper, it was noted that rigid SFM with

missing data can become ambiguous when the available points are in a degenerate configuration, such

as on a plane. To overcome this issue, the authors introduced the notion of
motion manifold
, which

constrains the recovered motion to be feasible. The available tracks can then be projected on this

manifold, which makes reconstruction robust to those degenerate scenarios. In
Paladini
et al.
[
2009
],

Search WWH ::

Custom Search