Information Technology Reference
In-Depth Information
a warped layer is then modeled as the number of pixels that vote for the layer. The
warped layers at
t
t
are updated by assigning the overlapping regions
only to the top layers, which yield the maximum similarity score.
Pixels in the uncovered regions are assigned separately by using the extracted
layers of previous time (
t
−
1and
t
−
Δ
−
1). For an uncovered pixel, the set of candidate current
time (
t
) layer labels corresponding to the previous time layer label at that pixel are
determined by using the estimated layer correspondences. Finally, the uncovered
pixel is assigned to the spatially closest candidate to avoid the creation of discon-
nected support maps and to ensure a maximum level of compactness.
A similar strategy is followed for uncovered pixels of the interpolation frame.
For each pixel of an uncovered region, the candidate layer set is initialized as the
neighboring layers of the region. Each pixel of an uncovered region, is then warped
to the previous time using the interpolated backward motion models of the candidate
layers. If the layer label at the warped location is different for a candidate layer, it is
removed from the set. Finally, as before, the spatially closest layer is selected among
the candidates for each uncovered pixel.
3.4
Middle Frame Interpolation
In the final stage of the algorithm, pixel intensity vectors at the middle, or inter-
polation, frame are estimated from the interpolated layer motion models and the
interpolated layer visibility images. For each pixel of layer
i
of the middle frame,
corresponding positions in the original frames
F
t
−
1
and
F
t
are computed using the
previously estimated backward and forward layer motion models
P
t
−
Δ
t
i
,
b
and
P
t
−
Δ
t
i
,
f
.
The interpolated intensity vector is modeled as a function of the intensities at the
corresponding sub-pixel locations in the original frames
F
t
−
1
and
F
t
.
Let the intensity vector (and layer label) at an integer pixel location
x
t
−
Δ
t
of the
middle frame
F
t
−
Δ
t
be denoted as
I
t
−
Δ
t
(
L
t
−
Δ
t
), and the intensity vectors (layer
labels) at the corresponding sub-pixel locations
x
t
−
1
in
F
t
−
1
and
x
t
in
F
t
are com-
puted to be
I
t
−
1
(
L
t
−
1
)and
I
t
(
L
t
), respectively. For the time being, assume that
at least one of the locations
x
t
−
1
and
x
t
falls inside the frame boundaries. Then,
the pixel intensity vector
x
t
−
Δ
t
is determined by the following piecewise linear
function:
⎧
⎨
I
t
−
1
:
L
t
−
Δ
t
=
L
t
−
1
,
L
t
−
Δ
t
=
L
t
I
t
:
L
t
−
Δ
t
=
L
t
−
1
,
L
t
−
Δ
t
=
L
t
I
t
−
Δ
t
=
:
L
t
−
Δ
t
=
L
t
−
1
,
L
t
−
Δ
t
=
L
t
,
(11)
t
)
I
t
−
1
+
t
I
t
⎩
(1
−
Δ
Δ
I
t
−
1
I
t
−
<
T
I
N
t
−
Δ
t
I
j
−
arg min
I
j
:else,
I
t
−
1
,
I
t
∈{
}
