The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 144

where X is a random variable, and x =( x i ) T is a random vector. Hence, as by
(7.51), q V ( V ) is a multivariate Gaussian with covariance matrix Λ V − 1 ,weget
E V ( v k v k )=Tr ( Λ V − 1 ) kk + v k T v k ,

(7.70)

where ( Λ V − 1 ) kk denotes the k th D V × D V block element along the diagonal of

Λ V − 1 .
Getting the moments of q ∗ W,τ ( W k ,τ k ) requires a bit more work. Let us first
consider

E W,τ ( τ k w kj ), which by (7.29) and the previously evaluated moments

gives

E W,τ ( τ k w kj )
= a τ k ,b τ k ) w kj , ( τ k Λ k ) − 1 )d w kj d τ k

τ k Gam( τ k | w kj N ( w kj |

= w kj a τ k ,b τ k )d τ k

τ k Gam( τ k |

a τ k
b τ k

w kj .

= (7.71)

E W,τ ( τ k w kj w kj )weget

For

E W,τ ( τ k w kj w kj )
= a τ k ,b τ k ) w kj , ( τ k Λ k ) − 1 )d w kj d τ k w kj w kj N τ k Gam( τ k | ( w kj |

= a τ k ,b τ k ) E W ( w kj w kj )d τ k τ k Gam( τ k |

= w kj T w kj E τ ( τ k )+Tr( Λ k − 1 )

a τ k
b τ k

w kj T w kj +Tr( Λ k − 1 ) . = (7.72)

E W,τ ( τ k w kj w kj ) can be derived in a similar way, and results in
E W,τ ( τ k w kj w kj )= a τ k
b τ k

w kj w kj T + Λ k − 1 . (7.73)

w kj x n ) 2 ), which we get by binomial
expansion and substituting the previously evaluated moments, to get

The last required moment is E W,τ ( τ k ( y nj −

w kj x n ) 2 ) E W,τ ( τ k ( y nj −

E τ ( τ k ) y nj − E W,τ ( τ k w kj ) T x n y nj + x n E W,τ ( τ k w kj w kj ) x n = 2

a τ k
b τ k

w kj T x n ) 2 + x n Λ k − 1 x n .

= ( y nj − (7.74)

Now we have all the required expressions to compute the parameters of the
variational posterior density.