On the Biological Plausibility of Artificial Metaplasticity - Foundations on Natural and Artificial Computation - page 120

long-term potentiation (LTP), whereas downregulation -decrementing inhibing-
is known as long-term depression (LTD). LTP and LTD are believed to be
fundamental to storage of memory in the brain and hence learning.
The induction of synaptic changes in the levels of neural activity is explained
[6] in Fig. 1. Metaplasticity can be represented as variations in curve elongation
with respect to the level of activity and implies a shift of the LTP threshold
according to the weight strength of the synapse [6]. Fig. 1 graphically illustrate
this idea. Understanding metaplasticity may yield new insights into how the
modification of synapses is regulated and how information is stored by synapses
in the brain. [7, 8, 9]
This paper is organized as follows. Section 2 provides a brief introduction to
related concepts (e.g. synaptic plasticity). Main postulation regarding the re-
lation between metaplasticity and Shannon's Information Theory is introduced
in Section 3, to ease the understanding of the proposed model. In Section 4,
general mathematical theory is applied to describe the proposed implementa-
tion of AMP in a MLP, whose learning process is based on error minimization.
In Section 5, we implement the AMP algorithm in the MLP neural network,
which is trained with the BP algorithm with a modified objective function.
Section 6 presents a discussion on the biological plausibility of the AMMLP al-
gorithm, refering to experimental results. Finally, Section 7 summarizes main
conclusions.
2 Synaptic Plasticity and Metaplasticity
Synaptic plasticity refers to the ecacy modulation of information transmis-
sion between neurons, being related to the regulation of the number of ionic
channels in synapses. Synaptic plasticity mechanisms involve both molecular
and structural modifications that affect synaptic functioning, either enhancing
or depressing neuronal transmission. They include redistribution of postsynaptic
receptors, activation of intracellular signaling cascades, and formation/retraction
of the dendrites [10]. The first model of synaptic plasticity was postulated by
Hebb and it is known as the Hebb rule [11].
In Fig.1, the effect of metaplasticity is illustrated. This graphic shows a family
of curves in which each curve indicates the biological variation in weight, Δω ,
respective of the neurons activation frequency or postsynaptic activity. If post-
synaptic activity is high, by metaplasticity property, the curve will move to the
right, reinforcing the LTP. Andina postulates [1] that high postsynaptic activity
corresponds to high frequent excitations -frequent input classes in an artificial
model-. In the same way, the left-hand side curves of the family corresponding to
low previous synaptic activity correspond to low frequent excitations produced
by patterns of unfrequent classes. During training, postsynaptic activity is the
same for each training pattern -one excitation by epoch-. As it can be observed
in Fig.1, for a given postsynaptic activity value, a higher Δω corresponds to the
un-frequent classes curves than to the curves corresponding to frequent ones.