Biomedical Engineering Reference
In-Depth Information
For a fixed number of neurons, capacity is not
bounded above:
MREM needs much less neurons to represent the
pattern than BH.
For a fixed number of possible outputs, capacity
is bounded below by a positive constant:
Suppose
N
fixed. Equation (4.2) can be rewrit-
ten in the following form:
Suppose
L
is fixed. Equation can be rewritten
as follows:
1
1)
N
− +
1
z
2
2
(
N L
,
)
≈
2
L
+
4(
N
− +
1)
z
+
2(
N
−
N z
2
2
L
4(
L
− +
1)
2
z
2
2(
L
−
1)
2
+ −
(
L
2)
z
2
(4.2)
1
(
N L
,
)
≈
2
+
+
Lz
2
N
N
2
= ∞
, since
the coefficient of
L
in this expression is positive.
What actually happens is that α(
N
,·), as
a function of
L
, has a minimum at the point
2
If we make
L
tend to ∞, we get
lim
(4.3)
(
N L
,
)
L
→∞
It can be easily seen that this expression rep-
resents a function whose value decreases as
N
grows. So, a net with more neurons than other,
and the same possible states, will present less
capacity than the second one.
Thus, a minimum positive capacity can be
computed for each possible value of
L
, verifying
= − − + ≈
for z
α
=2.326. It is a de-
creasing function for
L
<
L
0
(
N
) and increasing
for
L
≥
L
0
(
N
).
One consequence of this result is that, for ap-
propriate choice of
N
and
L
, the capacity of the
net can be α (
N,L
)>1.
This fact can be interpreted as a adequate
representation of the multi-valued information,
because, to represent the same patterns as MREM
with
N
and
L
fixed, BH needs
NL
binary neurons
and therefore the maximum number of stored
patterns may be greater than
N
. So it is not a
strange thing that the capacity can reach values
greater than 1, if the patterns are multi-valued,
L
0
(
N
)
(
N
1)(
N
1
z
)
N
a
2
>
.
(
L
)
=
lim
(
N L
,
)
=
0
min
2
Lz
N
→∞
α
min
(
L
) coincides with the asymptotic capac-
ity for the net with
L
possible neuron outputs.
For example, if
L
=2 (as in BH), an asymptotic
capacity of α
min
(2)=0.1847 is obtained, exactly the
capacity for BH provided in other works (Hertz
et al., 1991).
Figure 1. Many individual patterns are loaded into the net, forming a group of local minima of the energy
function (left). When the number of patterns to be loaded is greater than the capacity of the network,
the corresponding local minima are merged (right). The formation of these new local optima can be
interpreted as the apparition and learning of the associated concepts.
Search WWH ::
Custom Search