Graphics Reference
In-Depth Information
and deal with occlusions while estimating the flow. We will return to the role of
occlusions in dense correspondence in Section
19
.
5.3.4
Layered Flow
One way to get around a global smoothness assumption on the flowfield is to split the
image into regions called
layers
, and assume that the motion field within each layer
is smooth. This allows substantial discontinuities of the flow at layer boundaries. We
mentioned
layered motion
in Section
2.9
as a classical approach for video matting;
thesemethods grewout of early researchonvideoprocessing andoptical flow.We can
look at Figure
5.7
in a different way as an example of decomposing an observed image
(left side) into a pair of planar layers (right side), each of whichmoves independently.
Layered flowmethods are also related to the visual effects problemof post-converting
a monocular film into stereo.
Black and Anandan [
48
] proposed a classical layered motion approach using the
robust penalty functions of Section
5.3.3.3
to estimate a dominant motion in the
scene, classify inlier pixels into a solved layer, and re-apply the process to outlier
pixels. This approachmay not work well if there are a large number of roughly equal-
support motions in the scene, and there is no way to enforce cleanly segmented
layers.
Black and Jepson [
47
] proposed amore explicit layer-based optical flow algorithm.
The scene is assumed to be composed of multiple, independently moving planar
regions, so that the flow field in each region can be represented by a low-dimensional
(e.g., affine or quadratic) parametric transformation. First, a coarse optical flow field
is estimated for the entire image. A parametric model is then fit to the flow field in
each roughly-constant-intensity image region. Finally, an additional low-magnitude
local deformation field is estimated in each region to obtain the final optical flow
field. Weiss [
541
] proposed the concept of “smoothness in layers,” removing the
assumption that each layer had to be well fit by a parametric model and instead
requiring that the (non-parametric) flowfield in each layer simply be smooth. The EM
algorithm is used to alternate between estimating layer memberships and estimating
the flow field in each layer.
Historically, the best-performing optical flow algorithms have not used layered
motion models, instead using robust penalty functions and cross-checking to deal
with discontinuities. However, Sun et al. [
477
] recently proposed a competitive layer-
based method that builds on the previous two approaches with an advanced non-
parametric Bayesian graphical model.
5.3.5
Large-Displacement Optical Flow
While the hierarchical approach to optical flow estimation can handle certain cases
involving large values of
, additional techniques are often necessary to deal
with such
large displacements
. Brox et al. [
73
] noted that hierarchical differential
techniques (like Horn-Schunck and its modifications) are not able to detect the flow
of a structure whose size in the image is smaller than its motion vector magnitude.
For example, a person's hand may appear to be a few pixels wide but may move tens
of pixels between frames due to fast motion. In such cases, the small structure may
(
u
,
v
)
