ARTIFICIAL NEURAL NETWORKS FOR DATA MINING AND FEATURE EXTRACTION - Quantitative Medical Data Analysis Using Mathematical Tools and Statistical Techniques - page 324

An input neuron is a neuron which takes a single stimulus I as input
and returns an activation of the form x = (I) : A multi-layer perceptron
(MLP) has a input layer which is connected to one or more hidden layers,
with the last hidden layer being connected to an output layer. Subsequent
layers are completely connected, but there are no connections between two
neurons in the same layer or between neurons that are not in subsequent
layers. A 3-layer MLP has r input neurons connected to m neurons in a

Fig. 3. A Multi-Layer Perceptron

single hidden layer which are connected to n neurons in an output layer.
It has been shown that a 3-layer MLP can approximate any absolutely
integrable mapping of the type

f (I 1 ; : : : ; I r ) = (y 1 ; : : : ; y n )

to within any " > 0; where I j is the stimulus presented to the j th

input

neuron and y k is the activation from the k th output neuron 13 .
If we let x = (x 1 ; : : : ; x r ) denote the vector of activations from the input
to the hidden layer, then y j = (s j j ) ; j = 1; : : : n and

X

m

s j = jk (w k x k )

k=1

where w k = (w k1 ; : : : ; w kr ) denotes the vector of weights between the input
layer and the k th hidden neuron,denotes the standard inner product, and
jk denotes the weight between the k th hidden neuron and the j th output
neuron.