Biomedical Engineering Reference
In-Depth Information
squared are in another category. The situation becomes even more complex if there are more than two
classification categories. The problem is that the points in the right panel of Fig. 7.9 are not linearly
separable , meaning they can not be separated by a line.The most simple example of a logical function that
is not linearly separable is the XOR function shown graphically in Fig. 7.10. The problem of building
and trained neural networks capable of classifying nonseparable points nearly ended active research in
neural networks in the 1970s.
Linearly Separable
Not Linearly Separable
Figure 7.9: Example of linearly separable and not linearly separable data.
XOR
Figure 7.10: XOR logical function is no linearly separable.
7.4 MULTI-LAYEREDNETWORKS
The points in Figs. 7.9 and 7.10 points can be classified if many lines are used. For example, to classify
the XOR points, only two lines are needed. The practical meaning of adding more lines is the addition of
hidden layers of perceptions in the network as shown in Fig. 7.11. Each unit receives inputs only from the
preceding layer and can only send outputs to units in the layer ahead. Hidden layers allow for multiple
types of points, separated by nonlinear regions, to be grouped together. Furthermore, adding more hidden
layers, in principle allows for any complex set of regions to be defined.
7.4.1 Backpropagation
Initially there was no way to systematically tune the weights to separate any generic set of groupings.
Graphically, this is equivalent to changing the slope and intercept of many decision lines in a coordinated
way. It was not until the early 1980's that a method was found. First, an ideal target output, T (a vector
Search WWH ::




Custom Search