Information Technology Reference
In-Depth Information
Fig. 11
Building a patch of
multiscale coefficients, for a
single color channel image
(i.e band-pass or high-pass subbands of the multiscale transform), we build inter-
channel and interscale patches. To do so, in each color channel, we first group the
coefficients of closest scale and location (see Fig. 11):
k
,
p
=
w
I
c
1
,
p
;
w
I
c
k
,
p
,
w
I
c
k
,
p
+(1
,
0)
,
w
I
c
(1
,
0)
,
w
I
c
k
,
p
+(0
,
1)
,
w
I
c
(0
,
1)
,
w
I
c
(18)
k
,
p
−
k
,
p
−
k
−
and then build interchannel patches
W
k
,
p
by concatenating the patches of the three
color channels (YUV):
W
k
,
p
=
w
I
Y
k
,
p
.
k
,
p
,
w
I
U
k
,
p
,
w
I
V
(19)
With the approximation coefficients (i.e the low-pass subband of the multiscale
transform), we build interchannel and intrascale patches by concatenating across
channels the 3 by 3 neighborhoods of the low-frequency coefficients (making
patches of length 27). We denote by
W
k
,
p
either a low-pass or a high-pass or band-
pass patch. We use the Laplacian pyramid as the multiscale transform of the images
for its low redundancy and near invariance properties.
The multiscale patches description obtained is the set of all patches
W
k
,
p
for all
scales
k
and locations
p
. It is said to be sparse because 1) the set of patches of large
energy (sum of squared coefficients) is a small subset of the set of all multiscale
patches
W
k
,
p
}
0
≤
k
≤
K
−
1
,
p
∈
Z
and 2) this small subset describes well the content of
the image (this is a sparsity property inherited from the sparsity of the multiscale
decomposition: a small group yields a good representation). We select the so-called
sparse multiscale patches
by thresholding the energy level at each scale
k
and thus
obtain spatial descriptors of a frame of the video (see Section 3.3 for specific values
of thresholds).
{
3.1.2
Temporal Descriptors: GOP Motion Patches (GOP-MP)
To capture the motion information in a GOP, we also use the concept of patches built
on coherent information. Here, the coherence is sought through time: the patches are
made of motion vectors that follow motion through the GOP. One patch of the GOP
Motion Patches (
GOP-MP
) description captures the temporal coherence within the
GOP at a particular location
p
=(
x
,
y
) by encoding the motion of the block centered
