The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 136

where the constant represents all terms in (7.15) that are independent of W
and τ ,and E α are the expectations evaluated with respect to Z and α

respectively. This expression shows that q ∗ W,τ factorises with respect to k ,which
allows us to handle the q W,τ ( W k ,τ k )'s separately, by solving

E Z and

ln q ∗ W,τ ( W k ,τ k )=

n E Z ( z nk ln p ( y n |

W k ,τ k )) + E α (ln p ( W k ,τ k | α k )) + const.

(7.26)

Using the classifier model (7.7), we get

n E Z ( z nk ln p ( y n |

W k ,τ k ))

=

n E Z ( z nk )ln

w kj x n ,τ − k )

j N

( y nj |

1
2 ln τ k −

w kj x n ) 2 +const. =

n

r nk
j

τ k
2 ( y nj −

ln τ k + const.

= D 2

r nk

(7.27)

n

r nk y nj − 2 w kj
n

τ k
2

r nk x n y nj + w kj r nk x n x n

− w kj ,

j n n

where r nk ≡ E Z ( z nk )isthe responsibility of classifier k for observation n ,and y nj
is the j th element of y n . The constant represents the terms that are independent
of W k and τ k .
E α (ln p ( W k ,τ k |

α k )) is expanded by the use of (7.8) and results in

E α (ln p ( W k ,τ k | α k ))

=
j

E α ln a τ ,b τ )

0 , ( α k τ k ) − 1 I )+lnGam( τ k | N ( w kj |

D 2

b τ τ k +const.

=
j

τ k

2 E α ( α k ) w kj w kj +( a τ −

ln τ k − 1) ln τ k −

= D Y a τ −

ln τ k

D Y + D X D Y
2

⎛ ⎞

E α ( α k )
j

τ k
2

⎝ 2 D Y b τ +

w kj w kj

⎠ + const.

− (7.28)