Information Technology Reference
In-Depth Information
which controls the temporal penalization. These data-dependent kernel components
determine the steering action at sample
x
i
, and are described next.
Following [14], the spatial steering matrix
H
i
is defined by:
H
i
=
h
s
C
i
−
1
2
,
(25)
1
where
h
s
is a global spatial smoothing parameter, and
C
i
is a 2
2 covariance matrix
given by (18), which captures the sample variations in a local spatial neighborhood
around
x
i
.
C
i
is constructed in a parametric manner, as shown in (18).
The motion steering matrix
H
i
is constructed on the basis of a local estimate
of the motion (or optical flow vector)
m
i
=[
m
1
i
,
m
2
i
]
T
at
x
i
. Namely, we warp the
kernel along the local motion trajectory using the following shearing transformation:
(
x
1
i
−
×
x
1
)
←
(
x
1
i
−
x
1
)
−
m
1
i
·
(
t
i
−
t
)
t
)
.
−
←
−
−
·
−
(
x
2
i
x
2
)
(
x
2
i
x
2
)
m
2
i
(
t
i
Hence,
⎡
⎤
−
10
m
1
i
⎣
⎦
.
H
i
=
01
−
m
2
i
(26)
00
0
Assuming a spatial prototype kernel was used with elliptical footprint, this results
in a spatiotemporal kernel with the shape of a tube or cylinder with elliptical cross-
sections at any time instance
t
. Most importantly, the center point of each such
cross-section moves along the motion path.
The final component of (24) is a temporal kernel that provides temporal penal-
ization. A natural approach is to give higher weights to samples in frames closer to
t
. An example of such a kernel is the following:
h
t
exp
,
2
t
)=
1
−
|
t
i
−
t
|
K
h
t
(
t
i
−
(27)
2
h
t
where a temporal smoothing parameter
h
t
controls the relative temporal extent of
the kernel. We use the temporal kernel (27) in this section to illustrate the MASK
approach. However, we will introduce a more powerful adaptive temporal weighting
kernel in Section 4.2, which acts to compensate for unreliable local motion vector
estimates.
3.3
Spatial Upscaling and Temporal Frame Interpolation
Having introduced our choice of 3-D smoothing matrix,
H
3D
i
, using Gaussian kernel
for
K
, we have the MASK function as
