Information Technology Reference
In-Depth Information
z k = y k w
x k .
·
From the condition of correct classification and the definition of the perceptron
output, it can be shown that the perceptron output is correct if
z k > 0 .
If the norm of the weight vector is modified, without changing its orientation,
by multiplying all components w i (including w 0 ) by the same constant, the
perceptron output is the same. Only the hyperplane orientation, defined by the
unitary vector w /
, is relevant for the classification task. The properties
of the linear separation do not depend on the norm of w , but only on its
orientation.
w
6.3.3 Stability of an Example
In order to investigate the training properties of the neurons, we introduce
the concept of stability γ of an example
γ k = y k w
x k
·
w
= z k
w
.
Comparing with the distance to the separating hyperplane, and given that
y k = 1, the magnitude of the stability γ k is nothing but the distance of
the example k to the separating hyperplane. That is illustrated on Fig. 6.6 for
the case of real-valued inputs. In terms of stability, the condition for correct
classification can be written as
γ k > 0 .
Remark. The stabilities of the examples are a measure of our confidence in
the classification. We will see in the last part of this chapter that the proba-
bilistic interpretation of the classification is a function of those stabilities.
Some examples have interesting properties. The distance κ to the sepa-
rating hyperplane of the example of L M that is closest to the hyperplane is
called the margin . The region in input space on both sides of the hyperplane,
of width 2 κ , centered on the latter, does not contain any example.
Among all possible separating hyperplanes, the hyperplane with maximal
margin , also called optimal stability perceptron , has interesting properties. In
particular, it is robust with respect to small perturbations in the inputs or
the weights. The support vector machines, introduced later in this chapter,
are based on the concept of maximal margin.
Search WWH ::




Custom Search