Classification: Advanced Methods - Data Mining: Concepts and Techniques - page 398

9.2 Classification by Backpropagation
“ What is backpropagation? ” Backpropagation is a neural network learning algorithm.
The neural networks field was originally kindled by psychologists and neurobiologists
who sought to develop and test computational analogs of neurons. Roughly speaking, a
neural network is a set of connected input/output units in which each connection has
a weight associated with it. During the learning phase, the network learns by adjusting
the weights so as to be able to predict the correct class label of the input tuples. Neural
network learning is also referred to as connectionist learning due to the connections
between units.
Neural networks involve long training times and are therefore more suitable for appli-
cations where this is feasible. They require a number of parameters that are typically
best determined empirically such as the network topology or “structure.” Neural net-
works have been criticized for their poor interpretability. For example, it is difficult for
humans to interpret the symbolic meaning behind the learned weights and of “hidden
units” in the network. These features initially made neural networks less desirable for
data mining.
Advantages of neural networks, however, include their high tolerance of noisy data
as well as their ability to classify patterns on which they have not been trained. They
can be used when you may have little knowledge of the relationships between attributes
and classes. They are well suited for continuous-valued inputs and outputs, unlike most
decision tree algorithms. They have been successful on a wide array of real-world data,
including handwritten character recognition, pathology and laboratory medicine, and
training a computer to pronounce English text. Neural network algorithms are inher-
ently parallel; parallelization techniques can be used to speed up the computation
process. In addition, several techniques have been recently developed for rule extrac-
tion from trained neural networks. These factors contribute to the usefulness of neural
networks for classification and numeric prediction in data mining.
There are many different kinds of neural networks and neural network algorithms.
The most popular neural network algorithm is backpropagation , which gained repute
in the 1980s. In Section 9.2.1 you will learn about multilayer feed-forward net-
works, the type of neural network on which the backpropagation algorithm performs.
Section 9.2.2 discusses defining a network topology. The backpropagation algorithm is
described in Section 9.2.3. Rule extraction from trained neural networks is discussed in
Section 9.2.4.
9.2.1 A Multilayer Feed-Forward Neural Network
The backpropagation algorithm performs learning on a multilayer feed-forward neural
network. It iteratively learns a set of weights for prediction of the class label of tuples.
A multilayer feed-forward neural network consists of an input layer , one or more hidden
layers , and an output layer . An example of a multilayer feed-forward network is shown
in Figure 9.2.