Information Technology Reference
In-Depth Information
Fig. 1.29. Linear separation by a Perceptron (neural network with a single output,
without hidden neurons: 10% misclassification rate
Fig. 1.30. Separation by a network with a small number of hidden neurons. Three
examples per class are misclassified
Fig. 1.31. Separation by an overparameterized neural network. All examples are
correctly classified, but the generalization capacity is low
n -dimensional space,
n
v =
w i x i =0 .
i =1
Hence, v> 0 for all examples of one of the classes, and v< 0 for all examples
of the other class. Figure 1.29 shows a separation surface that can be defined
by a Perceptron, for the example of capacitors and integrated circuits.
Hidden neurons allow multilayer Perceptrons to define more complex sep-
aration surfaces, as shown on Fig. 1.30.
As usual, one can obtain zero misclassifications if enough hidden neurons
are added, but that is detrimental to generalization because of overfitting.
Fig. 1.31 illustrates a case of blatant overfitting.
When a multi-class problem is split into pairwise separation problems, lin-
ear separation between two classes is often complex enough; very frequently,
Search WWH ::




Custom Search