Biomedical Engineering Reference
In-Depth Information
The total cost function can be constructed in several ways from the instantaneous cost, de-
pending on the error function
f
(∙). If
f
(∙) is a quadratic function of the error, the total cost over a
data set of
N
samples is the sum of the instantaneous costs.
N
N
∑
= =
2
(
)
[
(
)
(
(
))]
(3.50)
J
=
g
n
⋅
d
n
−
y
χ
n
k
k
n
1
k
1
N
N
∏
= =
J
=
g
(
n
)
⋅
p
(
d
(
n
)
−
y
(
χ
(
n
)))
(3.51)
k
k
n
1
k
1
If
f
(∙) is a PDF in the error, the appropriate total cost function is a product of the instanta-
neous costs and returns us toward the mixture of experts when the gate is input based. We explain
now in more detail the hard competitive structure (shown in Figure
3.17
). The experts, in this case
predictors, are located on a spatial grid, analogous to a Kohonen's self-organizing map [
56
], and the
gate moderates the learning of the experts based on their distance from the best performing expert.
The best predictor is the one with the smallest local mean squared error
winner( ) argmin [
k
n
=
ε
k
n
( )]
(3.52)
where the local MSE is computed using the recursive estimate for each expert, repeated here for
convenience
2
ε
k
( )
λ
e
k
=
( )
+
(
1
λ
-
)
ε
k
n
1
(
-
)
0
λ
1
<
<
(3.53)
Neighborhood Update Map
Competitive Map
1−λ
e
2
ε
2
e
k
-
λ
H
k
(z)
+
( )
2
k
k
Error Power
Memory
+
x
(
n
)
d
k
,
k
*
k
*
k
*
z
+1
Winner
FIgURE 3.17:
Neighborhood map of competing predictors.
