Graphics Reference
In-Depth Information
dense correspondence across the whole image is to make the support of the window
large enough (for example, to make
large if we're using a Gaussian) to ensure
both eigenvalues are far from zero. Using large windows may not do a very good job
of estimating flow, since as we observed in Figure
4.11
, large square windows are
unlikely to remain square in the presence of camera motion.
On the other hand, the Lucas-Kanade algorithm is local, in that the flow vector
can be computed at each pixel independently, while the Horn-Schunck algorithm
is global, since all the flow vectors depend on each other through the differential
equations (
5.22
). This makes the Lucas-Kanade problem computationally easier to
solve. Since the linear system in Equation (
5.25
) also resulted from the assumption
that the flow vectors
σ
are small, a pyramidal implementation can be used in the
same way as in the previous section to handle large motions [
55
].
(
u
,
v
)
5.3.3
Refinements and Extensions
Many modern optical flow algorithms modify Equation (
5.21
) in some way — for
example, by changing the data term that measures deviation from the brightness
constancy assumption, the smoothness term that measures deviation from prior
expectations of realistic flow fields, or the formof the cost functions used to combine
the two terms. In this section, we briefly describe common modifications; many of
the best-performing algorithms combine several of these modifications.
5.3.3.1 Changes to the Data Term
The original Horn-Schunck data term encapsulates the assumption that pixel inten-
sities don't change over time, which isn't realistic for many real-world scenes. Uras
et al. [
506
] proposed instead the
gradient constancy assumption
that
∇
I
(
x
+
u
,
y
+
v
,
t
+
1
)
=∇
I
(
x
,
y
,
t
)
(5.26)
where
is the spatial gradient. This allows some local variation to illumination
changes; consequently, Brox et al. [
74
] proposed a modified data term reflecting
both brightness and gradient constancy:
∇
x
,
y
(
2
2
(5.27)
E
data
(
u
,
v
)
=
I
2
(
x
+
u
,
y
+
v
)
−
I
1
(
x
,
y
))
+
γ
∇
I
2
(
x
+
u
,
y
+
v
)
−∇
I
1
(
x
,
y
)
where
is around 100). Note that
Equation (
5.27
) directly expresses the deviation from the constancy assumptions,
insteadof using theTaylor approximation inEquation (
5.17
) that is only valid for small
u
and
v
. Xu et al. [
558
] claimed that it was better to use either the brightness constancy
or gradient constancy assumption at each pixel (but not both), and introduced a
binary switch variable for this purpose. An alternate approach is to explicitly model
an affine change in brightness at each pixel, as proposed by Negahdaripour [
346
] (in
which case these parameters also need to be estimated and regularized).
γ
weights the contribution of the terms (typically
γ
