Discrimination - Neural Networks: Methodology and Applications - page 334

Fig. 6.3. Schematic diagram of a perceptron

One element of choice between those approaches is the actual implementa-
tion of the application. Estimating probabilities on a digital computer is not
a problem. If the classifier will be implemented on a special-purpose device,
binary units can considerably improve the computation times and the com-
plexity of the circuits. A comparison between neural networks using binary
and real-valued neurons on an application of radar signals recognition may be
found in the Ph.D. thesis of Christelle Godin [Godin 2000].

6.2 Linear Separation: The Perceptron

The simplest network allowing the classification of data into two classes is a
single binary neuron. Introduced by Rosenblatt [Rosenblatt 1958], who called
it perceptron, it is shown schematically on Fig. 6.3. The output of Rosenblatt's
perceptron depends on the weighted sum of the input vector components x i ,
with weights w i ∈

R . This weighted sum, hereinafter called potential (as
in the previous chapters), is called field in the articles written by physicists
that studied the properties of neural networks. A Perceptron shares many
properties with the spins , which are elementary magnets. In particular, the
weighted sum plays the same role as the magnetic field in the context of the
physicists' models of magnetism.
Since the potential is a linear function of the inputs, the perceptron is
also called linear classifier. However, as already mentioned in Chap. 1, we
will consider generalized non-linear potentials, like polynomials (higher-order
neurons).
If the perceptron potential is larger than the neuron threshold s 0 ,theout-
put is σ = +1, otherwise σ =

1. Thus, the perceptron is a neuron whose
activation function is a threshold function. In Fig. 6.3 we followed the conven-
tion used in the previous chapters, and we included a constant input x 0 =1
with a weight w 0 . Its role is to shift the threshold of the activation function.
Now the input has an additional component x 0 = 1, so that we consider a

−