Digital Signal Processing Reference
In-Depth Information
1
2
D
where,
. It is clear that the information from all modalities are
integrated via the shared sparsity sparsity pattern of the matrices
Γ
=[
Γ
,
Γ
,···,
Γ
]
i
i
{
Γ
}
1
. This can
=
be reformulated in terms of kernel matrices as:
i
=
1
trace
(
Γ
D
)
+
λ
Γ
1
,
q
1
2
i
T
ˆ
i
i
Γ
=
arg min
Γ
K
X
i
,
X
i
Γ
)
−
2trace
(
K
X
i
,
Y
i
Γ
,
a
i
and
b
j
being
i
th
and
j
th
where, the kernel matrix
K
A
,
B
(
i
,
j
)=
φ
(
a
i
)
,
φ
(
b
j
)
columns of
A
and
B
respectively.
5.5.2
Composite Kernel Sparse Representation
Another way to combine information of different modalities is through composite
kernel, which efficiently combines kernel for each modality. The kernel combines
both within and between similarities of different modalities. For two modalities with
same feature dimension, the kernel matrix can be constructed as:
x
i
,
x
j
)+
α
2
κ
(
x
i
,
x
j
)+
α
3
κ
(
x
i
,
x
j
)+
α
4
κ
(
x
i
,
x
j
)
κ
(
X
i
,
X
j
)=
α
1
κ
(
,
x
i
being the feature
vector. It can be similarly extended to multiple modalities. However, the modalities
may be of different dimensions. In such cases, cross-simlarity measure is not pos-
sible. Hence, the modalities are divided according to being homogenous (e.g. right
and left iris) or heterogenous (fingerprint and iris). This is also reasonable, because
homogenous modalities are correlated at feature level but heteregenous modalities
may not be correlated. For
D
modalities, with
x
i
;
x
i
]
{
α
}
i
=
1
,···,
4
are weights for kernels and
X
i
=[
where,
i
{
}
i
∈S
j
,
S
⊆{
,
,···,
}
being
the sets of indices of each homogenous modality, the composite kernel for each set
is given as:
d
i
1
2
D
j
x
s
1
i
x
s
j
)
X
i
,
X
j
)=
∑
κ
(
κ
(
,
s
1
s
2
∈S
k
α
(5.25)
s
1
s
2
;
x
s
|S
k
|
i
x
s
1
i
;
x
s
2
i
Here,
X
i
=[
N
H
,
N
H
being
the number of different heterogenous modalities. The information from the different
heterogenous modalities can then be combined similar to sparse kernel fusion case:
;
···
]
,
S
k
=[
s
1
,
s
2
,···,
s
|S
k
|
]
and
k
=
1
,···,
i
=
1
trace
(
Γ
N
H
)
+
λ
Γ
1
,
q
1
2
i
T
ˆ
i
i
Γ
=
arg min
Γ
K
X
i
Γ
)
−
2trace
(
K
X
i
Γ
X
i
Y
i
,
,
where,
K
X
i
,
X
i
is
defined
for
each
S
i
as
in
previous
section
and
Γ
=
1
2
N
H
[
Γ
,
Γ
,···,
Γ
]
.
Search WWH ::
Custom Search