Graphics Reference
In-Depth Information
One advantage of the Horn-Schunck approach is that since Equation (
5.22
)is
fundamentally a diffusion equation, it will create good estimates of the dense cor-
respondence field even when the local gradient is nearly zero, smoothly filling in
reasonable flow vectors based on nearby locations where the flow is unambiguous.
That is, we avoid the
aperture problem
discussed in Section
4.1
, in which the corre-
spondence of flat or edge-like patches could not be reliably estimated. On the other
hand, the Taylor series approximation in Equation (
5.17
) is only valid when
is
close to zero, meaning that the method can only be applied when the images
I
1
and
I
2
are already very similar — that is, with
u
and
v
each less than one pixel.
This issue can be addressed using a
hierarchical
or
multiresolution
approach.
For example, Bergen et al. [
39
] recommended using a Laplacian pyramid (see
lier for the coarsest level of the pyramid (i.e., a low-resolution version of the original
pair). The coarsest level should be chosen so that the expected
(
u
,
v
)
is on the order
of a pixel at that level. Now we consider the next higher level of the pyramid. We
can't simply multiply the estimated
(
u
,
v
)
at each pixel by two and use these as ini-
tial guesses for the flow field at the next level, since the Taylor series expansion at
this higher level won't be valid for vectors with these magnitudes. Instead, we
warp
the second image plane toward the first using the estimated flow, so that the flow
between the first image and the warped second image is again expected to be nearly
zero. We continue this process as we go up the pyramid, computing the final optical
flow vectors as the sum of several small increments at each level. To summarize,
(
u
,
v
)
1. Create the Laplacian pyramids for
I
1
and
I
2
, indexed from 0 to
N
, where
N
is
the coarsest level of the pyramid. Initialize the working level
n
to
N
and let
I
1
and
I
2
be the images at the coarsest level of the pyramid, i.e.,
I
1
I
1
,
I
2
I
2
.
=
=
)
(
n
)
between
I
1
and
I
2
at level
n
.
2. Estimate the optical flow field
(
u
,
v
warp
as
k
=
n
2
k
−
n
+
1
)
(
k
)
— that is,
the sum of all the incremental motion fields so far, at the scale of level
n
3. Construct the optical flow field
(
u
,
v
)
(
u
,
v
1.
Since this generatesmotion vector estimates only for even
x
and
y
, use bilinear
interpolation to fill in motion vectors for the whole image.
4. Warp the pyramid image for
I
2
at level
n
−
warp
to create a new
−
1 using
(
u
,
v
)
image
I
2
. That is,
I
2
I
n
−
1
2
u
warp
,
y
v
warp
(
x
,
y
)
=
(
x
+
+
)
. Use bilinear interpolation
to sample pixels from
I
n
−
1
2
. Let
I
1
I
n
−
1
1
=
.
5.
Let
n
=
n
−
1.
6.
If
n
≥
0, go to Step 2. Otherwise, the final optical flow field is
(
u
,
v
)
=
k
=
0
2
k
)
(
k
)
.
(
u
,
v
Figure
5.5
shows an example of optical flow estimated using a hierarchical
implementation of the Horn-Schunck algorithm. Note that the algorithm produces
reasonable estimates even in flat regions.
Sun et al. [
475
] showed that applying a median filter to the incremental flow fields
during the warping process, as proposed by Wedel et al. [
538
], significantly improved
the resulting final flow field. It's also possible to apply incremental warping within
each level of the pyramid image in addition to between levels.
6
A Gaussian pyramid can also be used.
