The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 146

Expanding for the densities and substituting the variational moments results in

E W,τ,α (ln p ( W , τ | α ))

D X D Y
2

=
k

ψ

ln 2 π ( a α k ) ln b α k + ( a τ k ) ln b τ k −

− ψ −

⎛ ⎞

a α k
b α k

a τ k
b τ k

1
2

⎝

w kj T w kj + D Y Tr( Λ k − 1 )

⎠

− (7.78)

j

.

a τ k
b τ k

( a τ k ) ln b τ k ) + D Y − ln Γ( a τ )+ a τ ln b τ +( a τ − 1)( ψ − − b τ

The negative entropy E W,τ (ln q ( W , τ )) of { W , τ } isbasedon(7.29)and

results in

E W,τ (ln q ( W , τ ))

⎛
⎝
k

⎞
⎠

w kj , ( τ k Λ k ) − 1 )Gam( τ k | a τ k ,b τ k ) = E W,τ ln N ( w kj |

j

D 2

=
k

D 2

ln 2 π + 1 Λ k | E τ (ln τ k ) −

2 ln

|

j

E W,τ w kj )

τ
2 ( w kj −

w kj ) 2 Λ k ( w kj − ln Γ( a τ k ) + − −

b τ k E τ ( τ k ) + a τ k ln b τ k +( a τ k − 1) E τ (ln τ k ) −

a τ k −

(

= D Y
k

1+ D 2

D 2

( a τ k ) ln b τ k )

ψ − − (ln 2 π +1)

,

+ 1 Λ k |− ln Γ( a τ k )+ a τ k ln b τ k − a τ k

2 ln

| (7.79)

where the previously evaluated variational moments and
E W,τ

w kj ) =

τ
2 ( w kj −

1
2 D X

w kj ) 2 Λ k ( w kj −

− − (7.80)

was used.
We derive the expression

− E α (ln q ( α )) in combination, as that
allows for some simplification. Starting with

E α (ln p ( α ))

E α (ln p ( α )), we get from (7.17) and
(7.9), by expanding the densities and substituting the variational moments,

E α (ln p ( α )) (7.81)

.

=
k

b α a α k
b α k

( a α k ) ln b α k ) − ln Γ( a α )+ a α ln b α +( a α − 1)( ψ − −