The Optimal Set of Classifiers - Design and Analysis of Learning Classifier Systems - page 130

where p ( y n | x n , W k , τ ) is the model for the input/output relation of classifier k ,

parametrised by W = . Let us continue with the classifier

model, and then the model for the latent variables Z .

{ W k } and τ = { τ k }

7.2.2 Multivariate Regression Classifiers

The classifier model for classifier k is given by

W k x ,τ − 1
k

p ( y | x , W k ,τ k )= N ( y | I )

j =1 N

D Y

w jk x ,τ − 1 = ( y j | )

k

τ k
2 π

1 / 2

exp w kj x ) 2 ,

D Y

τ k
2 ( y j −

= − (7.7)

j =1

where y j is the j th element of y , W k is the D Y × D X weight matrix, and τ k is

the scalar noise precision. w kj is the j th row vector of the weight matrix W k .
This model assumes that each element of the output y is linearly related to x
with coecients w kj ,thatis, y j ≈ w kj x . Additionally, it assumes the elements

of the output vector to be independent and feature zero-mean Gaussian noise
with constant variance τ − k . Note that the noise variance is assumed to be the
same for each element of this output. It would be possible to assign each output
element its own noise variance estimate, but this model variation was omitted for
the sake of simplicity. If we have D Y = 1, we return to the univariate regression
model (5.3) that is described at length in Chap. 5.

7.2.3 Priors on the Classifier Model Parameters

Each element of the output is assumed to be related to the input by a smooth
function. As a consequence, the elements of the weight matrix W k are assumed
to be small which is expressed by assigning shrinkage priors to each row vector
w kj of the weight matrix W k . Additionally, the noise precision is assumed to
be larger, but not much larger than 0, and in no case infinite, which is given by
the prior Gam( τ k | a τ ,b τ ) on the noise precision. Thus, the prior on W k and τ k

is given by

D Y

p ( W k ,τ k | α k )= p ( w kj ,τ k | α k )

j =1

D Y

N

a τ ,b τ ) 0 , ( α k τ k ) − 1 I )Gam( τ k | = ( w kj | (7.8)

j =1

α k τ k
2 π

a τ τ k ,

D Y

D X / 2 b a τ τ ( a τ − 1)

exp

α k τ k
2

k
Γ( a τ )

w kj w kj −

= −

j =1