The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 147

The expression for E α (ln q ( α )) can be derived by observing that − E α (ln q ( α k ))

is the entropy of q α ( α k ). Thus, using q α ( α )= k q α ( α k ), substituting (7.39) for
q α ( α k ), and applying the entropy of the Gamma distribution [19], we get

ln Γ( a α k )

ln b α k + a α k

( a α k − ( a α k ) E α (ln q ( α )) = − − 1) ψ − (7.82)

k

Combining the above expressions and removing the terms that cancel out results
in

− E α (ln q ( α )) =
k

a α k ) ( a α k ) E α (ln p ( α )) − ln Γ( a α )+ a α ln b α +( a α − ψ

+lnΓ( a α k )+ a α k . (7.83) b α a α k

b α k

a α ln b α k − −

− E Z (ln q ( Z )) is also derived in combination
by using (7.12), (7.11) and (7.63), from which we get

The expression E Z,V (ln p ( Z | V ))

− E Z (ln q ( Z )) =
n

r nk ln g k ( x )

| v k = v k
r nk

E Z,V (ln p ( Z | V )) , (7.84)

k

where we have, as previously, approximated E V (ln g k ( x n )) by ln g k ( x n ) | v k = v k .

β )) is again based on simple expansion
of the distribution given by (7.18) and (7.13), and substituting the variational
moments, which results in

The derivation to get E V,β (ln p ( V |

E V,β (ln p ( V | β )) (7.85)

D V
2

v k T v k +Tr(( Λ V − 1 ) kk ) .

=
k

a β k
b β k

ψ

ln 2 π −

1
2

( a β k ) ln b β k − −

E V (ln q ( V )) by observing that it is the negative entropy of the Gaus-
sian (7.51), and thus evaluates to [19]

We get

1
2 ln

(1 + ln 2 π ) . + KD V

2

Λ V − 1 E V (ln q ( V )) = − | | (7.86)

As the priors on β k are of the same distribution form as the ones on α k ,the
expectations of their log-density results in a similar expression as (7.65) and is
given by

− E β (ln q ( β )) =
k

a β k ) ( a β k ) E β (ln p ( β )) − ln Γ( a β )+ a β ln b β +( a β − ψ

+lnΓ( a β k )+ a β k . (7.87)

a β k
b β k

a β ln b β k − − b β

This completes the evaluation of the expectations required to compute the
variational bound (7.75).