Graphics Reference
In-Depth Information
A
B
Figure 5.7.
Occlusions in optical flow occur when either the camera or objects in the scene
move between taking the images. In this sketch, occlusions are introduced due to a difference
in camera perspective. Point
A
in the left image is occluded in the right image, and Point
B
in the
right image is occluded in the left image.
(u
fwd
, v
fwd
)( )
(u
bwd
, v
bwd
)( )
I
1
I
2
Figure 5.8.
Cross-checking for detecting occlusions in optical flow. If the flow vector at the
white pixel in the left image points to the black pixel, then the flow vector at the black pixel
in the second image should point back at the white pixel. That is, the gray and white pixels
in image
I
1
should be the same. Otherwise, the correspondence is inconsistent and the pixel is
probably occluded in one of the images.
formerly occluded by objects in
I
1
will be revealed in
I
2
. Figure
5.7
illustrates a simple
example.
as
left-right checking
. That is, we compute the forward flow
fwd
from
I
1
to
I
2
(
u
,
v
)
bwd
from
I
2
to
I
1
. For a non-occluded pixel
and the backward flow
(
u
,
v
)
(
x
,
y
)
in
I
1
,we
should have that
fwd
bwd
u
fwd
v
fwd
(
u
,
v
)
(
x
,
y
)
=−
(
u
,
v
)
(
I
2
(
x
+
(
x
,
y
)
,
y
+
(
x
,
y
)))
(5.33)
That is, as illustrated in Figure
5.8
, the optical flow vector at the location given by
the forward flow in the second image should point back to the original pixel in the first
image. If the two vectors are not opposites, then the pixel is likely to be occluded in
one of the images. The “direction” of the occlusion can be determined by examining
the value of the cost function for each of the flows.
The robust cost functions discussed in the previous section can partially mitigate
the occlusion problem, but several proposed algorithms (e.g., [
556
]) explicitly detect
7
Belhumeur [
37
] notes that “half-occluded” regions is more accurate, since the problematic pixels
are visible in one image but not the other.
