The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 152

The first integral g k ( x ) q V ( v k )d v k is the expectation

E V ( g k ( x )) which does
not have an analytical solution. Thus, following Ueda and Ghahramani [216], it
is approximated by its maximum a-posteriori estimate

g k ( x ) q V ( v k )d v k ≈ g k ( x )

| v k = v k . (7.104)

The second integral q ∗ W,τ ( W k ,τ k ) W k x ,τ − k I )d W k d τ k is the expec-

tation E W,τ ( N ( y | W k x ,τ − k I )), that, by using (7.7) and (7.29), evaluates to

N ( y |

( y | W k x ,τ − k I )d W k d τ k E W,τ ( N

=

( y | W k x ,τ − 1

k

I ) q ∗ W |τ ( W k | τ k ) q τ ( τ k )d W k d τ k N

⎛
⎝
j

⎞

= ( y j | w kj x ,τ − 1 w kj , ( τ k Λ k ) − 1 )d w kj

⎠ q τ ( τ k )d τ k

N ) N ( w kj |

k

=
j

(1 + x T Λ k − 1 x ))Gam( τ k | ( y j | w kj T x ,τ − 1 a τ k ,b τ k )d τ k N

k

St y j | w kj T x , (1 + x T Λ k − 1 x ) − 1 a τ k
b τ k

, 2 a τ k , =

j

(7.105)

w kj T x , (1 + x T Λ k − 1 x ) − 1 a τ k /b τ k , 2 a τ k ) is the Student's t distribu-
tion with mean w kj T x , precision (1 + x T Λ k − 1 x ) − 1 a τ k /b τ k ,and2 a τ k degrees
of freedom. To derive the above we have used the convolution of two Gaussians
[19], given by

where St( y j |

( y j | w kj x ,τ − 1 w kj , ( τ k Λ k ) − 1 )d w kj N ) N ( w kj |

k

(1 + x T Λ k − 1 x )) ,

w kj T x ,τ − 1

( y j |

= N (7.106)

k

and the convolution of a Gaussian with a Gamma distribution [19],

(1 + x T Λ k − 1 x ))Gam( τ k | w kj T x ,τ − 1 ( y j | a τ k ,b τ k )d τ k N

k

=St y j | , 2 a τ k . (7.107) w kj T x , (1 + x T Λ k − 1 x ) − 1 a τ k

b τ k

Combining (7.103), (7.104), and (7.105) gives the final predictive density

p ( y | x , X , Y ) (7.108)

St y j | , 2 a τ k ,

=
k

| v k = v k
j

w kj T x , (1 + x T Λ k − 1 x ) − 1 a τ k

g k ( x )

b τ k

which is a mixture of Student's t distributions.