Information Technology Reference
In-Depth Information
Table 4.1. Notation used for the basic processing element shown in Figure 4.4.
a ijkl
-
activity after iteration t of feature k in layer l at position (j,j)
b tp
ijkl
potential of projection p that contributes to a ijkl
-
ψ kl
-
transfer function for feature k in layer l
φ kl
transfer function for projection p of feature k in layer l
-
v kl
weight of p th potential b tp
ijkl
that contributes to activity a ijkl
-
weight of q th activity a t
w pq
kl
i j k l that contributes to potential b tp
ijkl
-
number of potentials that contribute to activity a ijkl
P kl
-
T kl (t)
-
source access mode of projection klp : direct of buffered
L kl
-
source layer of projection klp
K pq
kl
-
source feature index of weight klpq
J pq
kl (j)
-
source row of weight klpq , depends on destination row j
I pq
kl (i)
-
source column of weight klpq , depends on destination column i
mapping of coordinates from layer l to layer l
Υ ll (x)
-
J p kl
-
row offset of weight klpq
I p kl
-
column offset of weight klpq
approximation of the continuous-time dynamics of neurons. Another example is
the modeling of synapses by a single multiplicative weight, as compared to a dy-
namical system that describes facilitation and depression of the synaptic efficacy.
Furthermore, in contrast to cortical neurons that produce action potentials, the basic
processing element outputs a graded response. These simplifications were made to
make network simulations feasible, despite the fact that typical Neural Abstraction
Pyramid networks will contain thousands to millions of such processing elements.
More complex processing elements might be more powerful, but they induce higher
computational costs and need more parameters for their description.
The basic processing element is already quite powerful. This can be seen from
the success of feed-forward neural networks with a single hidden layer. When
equipped with a large-enough number of projection units that have sigmoidal trans-
fer functions it can approximate, with arbitrary accuracy, any continuous function
of its inputs [48]. However, in typical networks the number of projection units will
be small and the computed functions will be simple. The weights that determine the
behavior of the network can be changed through learning. If the transfer functions
ψ kl and φ kl are differentiable, partial derivatives of the output error with respect to
a weight can be computed by backpropagation and gradient descent techniques can
be applied to adjust the weights.
4.2.2 Shared Weights
Where do the inputs a t
i j k l to a projection p of a feature cell ijkl come from?
All weights of a projection originate in the same layer. This source layer is called
Search WWH ::




Custom Search