Image Processing Reference
In-Depth Information
Δf
=
f
(
x
k
(
t
1
)
,y
k
(
t
1
)
,t
1
)
−
f
(
x
k
(
t
0
)
,y
k
(
t
0
)
,t
0
)) = 0
(12.106)
so that the substitution of Eq. (12.105) yields
DFD:
Δf
=
f
(
x
k
,y
k
,t
1
)
−
f
(
x
k
−
v
x
,y
k
−
v
y
,t
0
)=0
(12.107)
The spatial coordinates
x
k
,y
k
refer to those at time
t
1
, i.e.,
x
k
(
t
1
)
,y
k
(
t
1
). The differ-
ence function
Δf
is also called the
displaced frame difference
DFD, in the literature
because one moves a frame towards another using a displacement. In practice, how-
ever, the DFD is not exactly zero because the BCC is satisfied only approximately.
The problem is instead reformulated so that the
L
2
norm of the DFD is minimized
over
v
:
2
=
k
v
y
,t
0
))
2
=
k
f
k
)
2
(12.108)
Δf
(
f
(
x
k
,y
k
,t
1
)
−
f
(
x
k
−
v
x
,y
k
−
(
f
k
−
where
f
k
and
f
k
are the sampled image frame at time
t
1
and the sampled translated
image frame
11
at time
t
0
. The minimization is achieved by assuming that
f
is a local
patch, typically a 7
<C
, typically
C
=7. Thus, a typical implementation would search for a point yielding the least
×
7 square, and the displacement is limited,
|
v
|
2
by varying
v
for a local
f
with the size of 7
Δf
×
7 in a search window of a
15
15. Equation (12.108) measures the distance between two images after one is
translated towards the other. Altenatively, one can compare the frame to its translated
previous frame by using a similarity measure. Defining the discrete gray values in the
2D image patches as vectors i.e.,
×
f
=(
f
k−
1
, f
k
, f
k
+1
···
)
T
,
)
T
,
(12.109)
f
=(
···f
k−
1
,f
k
,f
k
+1
···
and
···
the Schwartz inequality,
f
,
f
f
T
f
cos(
θ
)=
|
|
|
|
=
f
≤
1
(12.110)
f
f
f
can be used used as a similarity measure. One can search for the pattern
f
that is most
parallel to
f
among patterns in its vicinity by maximizing cos(
θ
), which is insensitive
to multiplicative changes of
f
. Seeking the optimal
f
by maximizing cos(
θ
) requires
repetitive scalar products, which is also a correlation. When
f
holds, opti-
mizing the objective functions in Eq. (12.108) and Eq. (12.110) yield identical
v
as
is seen by the expansion
f
=
f
2
=
f
−
f
,
f
−
f
=
f
2
+
f
2
−
2
f
T
f
f
−
(12.111)
which is minimized at the same time as
f
T
f
is maximized. This is why using both
variants of the objective functions is known as correlation-based optical flow. A ma-
jor difference is, however, that the ratio cos(
θ
) is less affected by brightness changes
because these are approximated well by multiplicative amplifications of images.
11
This is also referred to as the
warped image
.
