Biomedical Engineering Reference
In-Depth Information
4. The generalization ability is insufficient.
5. The ability to separate essential parts in a complex sensory field (analytic ability)
is insufficient.
These points should be revised in the context of modern computer capabilities.
Currently, computers cannot implement neural networks comparable with the
human brain, which contains many billions of neurons, but it is possible to simulate
the neuron structures containing up to, and in some cases larger than, a million
neurons. In this case, it is interesting to know how the number of associative neurons
influences Rosenblatt perceptron performance.
We studied and described several modifications of Rosenblatt perceptrons and
experiments with them (O. Makeyev participated in this investigation) [
12
-
19
].
These experiments show that it is possible to overcome the above-mentioned
problems using modern hardware. In the experiments, the number of associative
neurons was changed from 1,000 to 512,000. The proposed perceptrons were tested
on a benchmark MNIST data set for handwritten digits recognition [
20
,
21
]. The
performance of the modified Rosenblatt perceptron, having 512,000 neurons, is
99.2% on this database. As computer technology improves, larger capacity recog-
nizers become feasible and higher recognition rates become possible. There are
data about different classifiers' performances on this database. The best classifier on
this database shows 99.3% [
21
].
Bernard Widrow was working along similar lines using systems known as
Adalines (ADAptive LINear Element - a single processing unit with threshold
non-linearity) [
22
,
23
]. Widrow, along with his graduate student M. Hoff,
proposed the Widrow/Hoff learning law or delta rule for neural network
training. In Widrow learning, the goal is to find the best possible weight vector
(for a very simple type of processing element) in terms of a least mean squared
error performance function criterion. This learning rule is one of the most
powerful and guarantees finding this optimum weight vector from any starting
point.
But in 1969, Marvin Minsky and Seymour Papert attacked neural network
research. They used predicates to describe the perceptron work [
24
]. In particular,
the following points are critical remarks concerning perceptron functioning:
1. The idea of thinking about classes of geometrical objects as classes of
n
-dimensional vectors (
a
1
,
,
a
n
) loses the geometric individuality of the
patterns and leads only to a theory that can do little more than
count
the number
of predicates.
2. Little attention has been paid to the size, or more precisely, the information
content, of the parameters (
a
1
,
...
,
a
n
). Some examples exist where the ratio
of the largest to the smallest of the coefficients is meaninglessly large. In
some cases, the information capacity needed to store
a
1
,
...
,
a
n
is even
greater than that needed to store the whole class of figures defined by the
pattern.
3. Closely related to the previous point is the problem of time of convergence in a
learning process.
...
