Image Processing Reference

In-Depth Information

1400

1400

1200

1200

1000

1000

800

800

600

600

400

400

200

200

0

0

0

200

400

600

800

1000

1200

1400

0

200

400

600

800

1000

1200

1400

Figure 4.1:
Singular values for a 5 frames sequence under perspective projection based on
Salzmann
et al.

[
2007b
]. Left: Without temporal consistency constraints between frames, the linear system is ill-

constrained. Right: Bounding the frame-to-frame displacements transforms the ill-conditioned linear

system into a well-conditioned one. The smaller singular values have increased and are now clearly non-

zero. Since our motion model introduces more equations than strictly necessary, the other values are also

affected, but only very slightly.

4.1

IMPOSINGTEMPORAL CONSISTENCY

When dealing with video sequences, one can assume that the surface does not move randomly be-

tween consecutive frames, whatever its physical properties. One way to overcome the rank deficiency

of the matrix of Eq.
4.1
is therefore to perform the reconstruction over several frames simultaneously.

This amounts to stacking the coordinate vectors
x
of Eq.
4.1
, one for each time frame, and creating

a block diagonal matrix whose elements are matrices
M
, again one for each time frame. Without

temporal constraints to link the coordinate vectors across frames, this system is just as ill-conditioned

as before. However, because displacement speeds are limited, the range of frame-to-frame motion

is always bounded, which can be expressed as a set of additional linear constraints of the form

x
t
−
1

x
t

−

=

≤
t
≤
N
f
,

0
,
2

(4.2)

where
x
t
is the coordinate vector for frame
t
and
N
f
is the total number of frames. These constraints

link the coordinate vectors and can be added to the correspondence equations in the joint system for

all
N
f
frames. The resulting linear system is much better-conditioned as depicted by Fig.
4.1
. Since

this system is solved in the least-squares sense, the motion equations will not be truly enforced, and

thus some motion will be allowed. As a result, given the shape at the beginning and at the end of

a sequence, the surface can be simultaneously reconstructed over the whole sequence as shown in

Fig.
4.2
.

These simple temporal constraints, however, do not accurately model the true dynamical

behavior of a non-rigid surface and, as a result, the reconstructions are not necessarily very accurate.

Furthermore, as discussed above, solving the linear system of Eq.
4.1
in the least-squares sense is

not strictly equivalent to minimizing the true reprojection error. In
Salzmann
et al.
[
2007a
], this was

remedied by exploiting techniques proposed for rigid object modeling
Kahl
[
2005
],
Ke and Kanade

Search WWH ::

Custom Search