Information Technology Reference
In-Depth Information
Fig. 6 Illustration of video processing based on motion-assisted spatiotemporal steering ker-
nel (MASK) regression
An overview of the proposed video interpolation and denoising algorithm based
on motion-assisted spatiotemporal steering kernel regression is provided in Fig. 6.
The algorithm estimates spatial and motion steering parameters using gradient-
based techniques. Hence, we first compute initial estimates of the spatial and tem-
poral derivatives, e.g. based on classic kernel regression. In this work, we obtain a
quick and robust estimate of the spatial orientation angle (
θ
i ), elongation (
ρ
i )and
scaling (
i ) parameters at x i by applying a vector quantization technique to the co-
variance matrix obtained from the spatial gradient data. This will be described in
Section 4.3. Motion vectors are estimated using the well-known Lucas and Kanade
method, based on both spatial and temporal gradients in a local region. This is fol-
lowed by computing estimates of the temporal motion reliability (
γ
), and is de-
scribed further in Section 4.2. Given spatial and motion steering parameters, final
MASK regression is applied directly on the input video samples; further details of
this step are provided in Section 4.4.
The following are further salient points for our algorithm based on MASK. We
first summarize them, and then provide details in subsequent subsections.
η
Block-by-Block Processing
Since the kernel-based estimator is a pointwise process, it is unnecessary to store
the orientations and motion vectors of all the pixels in a video frame ( H i and H i
for all i ) in memory. However, strict pixel-by-pixel processing would result in a
large number of redundant computations due to the overlapping neighborhoods
of nearby pixels. In order to reduce the computational load while keeping the
required memory space small, we break the video data into small blocks (e.g.
8
8 pixels), and process the blocks one-by-one.
Adaptive Temporal Penalization
MASK relies on motion vectors, and the visual quality of output video frames
is strongly associated with the accuracy of motion estimation. Even though our
motion estimation approach is able to estimate motion vectors quite accurately,
the estimated vectors become unreliable when the underlying scene motion and
×
 
Search WWH ::




Custom Search