Information Technology Reference
In-Depth Information
By combining (4.14) and (4.17), the term to maximise in the M-step of the
EM algorithm becomes
N
K
N
K
D Y
r nk ln p ( y n |
x n , w k )=
r nk
y nj ln w kj ,
n =1
n =1
k =1
k =1
j =1
under the constraint j w kj =1forall k . Considering each expert separately,
expert k has to solve the constraint optimisation problem
N
D Y
max
w k
r nk
y nj ln w nj ,
(4.18)
n =1
j =1
D Y
subject to
w kj =1 .
j =1
While the concepts introduced in the following sections are valid for any form
of expert models, a detailed description of how to train the above expert models
to find its parameters is given in Chap. 5.
4.3
Generalising the MoE Model
The standard MoE model assumes that each observation was generated by one
and only one expert. In this section, the model will be made more LCS-like by
replacing the term “expert” with “classifier”, and by introducing the additional
assumption that a classifier can only have produced the observation if it matches
the corresponding input. The following sections implement this assumption and
discuss its implications.
4.3.1
An Additional Layer of Forced Localisation
Let us recall that for a certain observation ( x , y ), the latent variable z determines
which classifier generated this observation. The generalisation that is introduced
assumes that a classifier k can have only generated this observation, that is,
z k = 1, if it matches the corresponding input.
Let us introduce an additional binary random vector m =( m 1 ,...,m K ) T ,
each element being associated with one classifier 2 . The elements of m are 1 if
and only if the associated classifier matches the current input x ,and0otherwise.
Unlike z , m does not comply to the 1-of- K structure, as more than one classifier
can match the same input. The elements of the random vector are linked to the
matching function by
p ( m k =1
|
x )= m k ( x ) ,
(4.19)
2 While the symbol m also refers to the matching function, its use as either the
matching function or the random variable that determines matching is apparent
from its context.
 
Search WWH ::




Custom Search