Image Processing Reference
In-Depth Information
differential is given by the difference between the two pixels along the two faces of the
cube, as
(
P
(1)
+
P
(1)
+
P
(1)
+
P
(1)
) -
xy
,
xy
+1,
xy
,
+1
xy
+1,
+
1
(
P
(0)
+
P
(0)
+
P
(0)
+
P
(0)
)
(4.94)
xy
,
xy
+1,
xy
,
+1
xy
+1, +1
∇
t
xy
=
,
8
Note that if the spacing between the images is other than one unit, this will change the
denominator in Equations 4.93 and 4.94, but this is a constant scale factor. We also need
means to calculate the averages. These can be computed as
u
+
u
+
u
+
u
u
+
u
+
u
+
u
x
-1,
y
x y
,
-
1
x
+1,
y
y
+
1
x
-1,
y
-
1
x
-1,
y
+
1
x
+1,
y
-
1
x
+1,
y
+
1
u
=
+
xy
,
2
4
v
v
v
v
v
v
v
v
+
+
+
+
+
+
xy
-1,
xy
,
-
1
xy
+1,
y
+
1
xy
-1,
-
1
xy
-1,
+
1
xy
+1,
-
1
xy
+1,
+
1
v
=
+
xy
,
2
4
(4.95)
The implementation of the computation of optical flow by the iterative solution in Equation
4.92 is presented in Code
4.21
. This function has two parameters that define the smoothing
parameter and the number of iterations. In the implementation, we use the matrices
u
,
v
,
uu
and
vv
to store the old and new estimates in each iteration. The values are updated according
to Equation 4.92. Derivatives and averages are computed by using Equations 4.93, 4.94
and 4.95. In a more elaborate implementation, it is convenient to include averages as we
discussed in the case of single image feature operators. This will improve the accuracy and
will reduce noise. Additionally, since derivatives can only be computed for small
displacements, generally, gradient algorithms are implemented with a hierarchical structure.
This will enable the computation of displacements larger than one pixel.
Figure
4.38
shows some examples of optical flow computation. In these examples, we
used the same images as in Figure
4.37
. The first row in the figure shows three results
obtained by different numbers of iterations and a fixed smoothing parameter. In this case,
the estimates converged quite quickly. Note that at the start, the estimates of flow in are
quite noisy, but they quickly improve; as the algorithm progresses the results are refined
and a more smooth and accurate motion is obtained. The second row in Figure
4.38
shows
the results for a fixed number of iterations and a variable smoothing parameter. The
regularisation parameter controls the compromise between the detail and the smoothness.
A
high
value of
will enforce the
smoothness
constraint whereas a
low
value will make the
brightness
constraint dominate the result. In the results we can observe that the largest
vectors point in the expected direction, upwards, whilst some of the smaller vectors are not
exactly correct. This is because there is occlusion and some regions have similar textures.
Clearly, we could select the brightest of these points by thresholding according to magnitude.
That would leave the largest vectors (the ones which point in exactly the right direction).
Optical flow has been used in automatic gait recognition (Huang, 1999; Little, 1998),
amongst other applications, partly because the displacements can be large between successive
images of a walking subject, which makes the correlation approach suitable. Figure
4.39
shows the result for a walking subject where brightness depicts magnitude (direction is not
shown). Figure
4.39
(a) shows the result for the differential approach, where the flow is
clearly more uncertain than that produced by the correlation approach shown in Figure
4.38
(b).
Another reason for using the correlation approach is that we are not concerned
λ
