Method - Protein Homology Detection Through Alignment of Markov Random Fields - page 20

Fig. 2.2 Illustration of probabilistic neural networks (PNN), in which ci i and c j are the protein
sequence features centered at the ith and jth residues, respectively, and m ij (mutual information) is
the feature that must be fetched at the ith and jth residues simultaneously

X X

G 1
g 1 ¼ 1 h

G 2
g 1 ; g 2 h

0 1 2

L h d k ; ð x k Þ¼

d k ; g 1 h

ð

g 1 ; g 2 h

ð \ h

g 2 ;

x k [ ÞÞ ð 2 : 3 Þ

where G 1 and G 2 are the number of gates in the two hidden layers, < · , · > denotes the

2
g 2

inner product of two vectors, h is the weight factor of the g 2 th neuron in the

1
g 1 ; g 2

second layer; h is the weight connecting the first layer and the second layer.

0
d k ; g 1

h first layer and the labels.

In current implementation, our neural network contains two hidden layers. The

is the weight connecting the

first hidden layer (i.e., the layer connecting to the input layer) contains 100 neurons,
and the second hidden layer (i.e., the layer connecting to the output layer) has 40
neurons. This neural network is similar to what is used by the Zhou group [ 6 ] for
inter-residue contact prediction, which uses 100 and 30 neurons in the two hidden
layers, respectively. The Zhou group has shown that using two hidden layers can
obtain slightly better performance than using a single hidden layer. The input layer
of our network has about 600 features, so in total, our neural network has between
60,000 and 70,000 parameters to be trained. We use the maximum likelihood
method to train the model parameter and to determine the number of neurons in

each hidden layer by maximizing the occurring probability of native C a

h

C a
distance in a set of training proteins. It is challenging to maximize the objective
function Eq. ( 2.2 ) since it is non-convex and a large amount of training data is used.
We use a limited-memory BFGS method [ 7 ] to ful

ll this. We generated an initial