The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 148

To simplify the computation of the variational bound, we define

L k ( q )= E W,τ,Z (ln p ( Y | W k ,τ k , z k )) + E W,τ,α (ln p ( W k ,τ k | α k ))

+ E α (ln p ( α k )) − E W,τ (ln q ( W k ,τ k )) − E α (ln q ( α k )) , (7.88)

which can be evaluated separately for each classifier by observing that all ex-
pectations except for

E V (ln q ( V )) are sums whose components can be evaluated
independently for each classifier. Furthermore,

L k ( q ) can be simplified by using

the relations

D X D Y
2

= a α k − a α , (7.89)

⎛ ⎞

a τ k
b τ k

1
2

⎝

w kj T w kj + D Y Tr( Λ k − 1 )

⎠ = b α k −

b α , (7.90)

j

which results from (7.40) and (7.41). Thus, the final, simplified expression for
L k ( q ) becomes
L k ( q )= D 2

ln 2 π
n

ψ

r nk + D X D Y
2

( a τ k ) ln b τ k −

−

r nk a τ k

2 + D Y x n Λ k − 1 x n

2
n

1

W k x n −

b τ k

y n −

a α k ln b α k + D 2 Λ k − 1 ln Γ( a α )+ a α ln b α +lnΓ( a α k ) − − ln | |

b τ a τ k
b τ k

a τ k ) ( a τ k ) a τ ln b τ k − + D Y − ln Γ( a τ )+ a τ ln b τ +( a τ − ψ −

+lnΓ( a τ k )+ a τ k .

(7.91)

All leftover terms from (7.75) are assigned to the mixing model, which results

in

L M ( q )= E Z,V (ln p ( Z | V )) + E V,β (ln p ( V | β )) + E β (ln p ( β ))

− E Z (ln q ( Z )) − E V (ln q ( V )) − E β (ln q ( β )) . (7.92)

We can again derive a simplified expression for L M ( q ) by using the relations

D V
2

= a β k − a β , (7.93)

Tr ( Λ V − 1 ) kk + v k T v k = b β k −

1
2

b β , (7.94)

which result from (7.56) and (7.57). Overall, this leads to the final simplified
expression
L M ( q )=
k

−

a β k ln b β k ln Γ( a β )+ a β ln b β +lnΓ( a β k )

− (7.95)

+
n

r nk ln g k ( x n ) ln r nk + 1

+ KD V
2

Λ V − 1 | v k = v k −

2 ln

| | .

k