Biomedical Engineering Reference
In-Depth Information
R
k
=
N
-
1
∑
N
h
k
( )
χ
n
1
(
-
)
χ
T
n
1
(
-
)
(3.43)
∑
n
=
1
h
k
(
)
n
=
1
1
N
∑
p
k
=
h
k
( )
χ
n
1
(
-
)
x
( )
(3.44)
N
∑
h
k
(
)
n
=
1
n
=
1
Note that the solution represents a weighted Wiener filter, where the data are weighted by
the posterior probabilities. If the experts are nonlinear, then the error used to iteratively train each
expert is simply interpreted as being weighted by the posterior probabilities,
h
k
(
n
). However, Zeevi
et al. [
52
] have shown that the nonlinearity of the gate allows the mixture of experts to be a universal
approximator, even if the expert predictors are linear.
No matter whether the experts are linear or nonlinear, there is an exact solution for the
Gaussian covariance parameter
σ
at the end of each epoch
N
1
∑
~
2
2
σ
=
h
(
n
)[
x
n
)
−
x
(
n
)]
(3.45)
k
N
k
∑
n
=
1
h
(
n
)
k
n
=
1
3.2.3.1 gated Competitive Experts.
A gated competitive expert model is characterized by several
adaptable “experts” that compete to explain the same data. That is, they all see the same input, and
all attempt to produce the same output. Their performance is both monitored and mediated by a
gate, whose goal is to determine the relative validity of the experts for the current data and then to
appropriately moderate their learning. The history of the gate's decisions can be used to segment the
time series.
In the activation phase, the total system output is a simple weighted sum of the experts'
outputs
K
∑
=
y
(
n
)
=
g
(
n
)
⋅
y
(
χ
(
n
))
(3.46)
k
k
k
1
where
y
k
is the output of the
k
th expert with χ as input and
g
k
is the
k
th output of the gate. The
input, χ(
n
) is a vector of delayed versions of the time series, in essence the output of a tapped delay
line, repeated here for convenience
T
χ
( )
=
x
( )
x n
(
-
1
) …
x n M
(
-
+
1
)
(3.47)
For the mixture to be meaningful, the gate outputs are constrained to sum to one.
