Geoscience Reference
In-Depth Information
Layer of
input
nodes
Layer of
hidden
PEs
Layer of
output
PEs
Layer of
input
nodes
Layer of
hidden
PEs
Layer of
output
PEs
(a)
(b)
FIGURE 13.3
Feedforward CNN architectures with one hidden layer: (a) fully connected and (b) partially
connected. (From Fischer, M.M.,
Environ. Plann. A
, 30(10), 1873, 1998.)
one hidden layer back to another or even from one node back to itself. The exact nature of each
full, or partial, recurrent structure will in these instances have a profound impact on the training
programme and learning capabilities of the CNN which, in turn, will affect its overall performance.
Moreover, in contrast to feedforward networks, the computational processing is not defined in a
unique manner according to a set of simple weighted connections because the temporal dimension
must now also be considered. When the output of a PE is fed back to the same element, we are
also dealing with a recursive computation that has no explicit halting condition. So, at a particular
instance in time, how do we tell if the fixed point of the recursive evaluation is the desired result
or just one of a set of intermediate computations? To help solve this problem, it is usual to assume
that each computation at each node will take a certain amount of time to process. If the arguments
for a PE have been transmitted at time
t
, then its output will be produced at time
t
+ 1. A recursive
process can therefore be stopped after a certain number of steps and the last computed output taken
as the result of its recursive computation.
13.6 LEARNING IN A COMPUTATIONAL NEURAL NETWORK
In addition to the information processing characteristics associated with their individual elements
and between-network differences that arise from the use of alternative network topologies, the learn-
ing or training process forms another important distinguishing feature of CNNs. In the context of
CNN learning, the process of training is perhaps best viewed as being a (typically) local, stepwise,
steepest-gradient-based search procedure. It is operating within a multidimensional weight space
and is looking for a solution (i.e. an ideal set of weights) which optimises a pre-specified objective
function with or without constraints (using dedicated performance criterion to evaluate each model).
Learning is performed in a progressive manner and is in most cases accomplished using an adaptive
procedure referred to as the
learning rule
,
training rule
or (machine)
learning algorithm
.
Standard practice is to distinguish between two different types of learning situation:
1. Supervised learning problems
2. Unsupervised learning problems
