Biomedical Engineering Reference
In-Depth Information
/UTPUTS
(IDDEN
,AYER
)NPUTS
Figure 7.11:
Generic feedforward multilayer network.
of outputs), is defined that corresponds to known inputs. Given an actual output,
a
j
, an error can be
computed between the actual and target output. The outputs can then be adjusted by tuning the weights
in a direction such that the sum of squared output errors,
E
p
, will be minimized.
2
T
a
j
2
1
E
p
=
−
.
(7.7)
Substitution of Eq. (7.4) into Eq. (7.7) yields
2
N
1
2
E
p
=
T
−
w(i, j)a
i
.
(7.8)
i
=
1
Another interpretation of
E
p
is that it defines an
energy function
or energy surface. The surface is at a
minimum when the error between
a
i
and
T
is smallest. In theory we might imagine the error being
zero, however, this generally not possible. Finding a practical solution is therefore is equivalent to finding
the set of weights that minimizes Eq. (7.8). For an arbitrary case there is no analytical way to find the
minimum. Instead the most common approach is the iterative method of
gradient descent
. First a function
is defined that characterizes how the energy surface (
E
p
) will change if the weights (
w
) change.
T
g
w(i, j)a
i
g
w(i, j)a
i
a
i
dE
p
dw
=
−
(7.9)
which is simply the derivative of Eq. (7.8) with respect to
w
. Evaluating Eq. (7.9) at a point gives the
slope of the energy surface with respect to
w
. The negative of this multidimensional slope is the direction
w
should be changed to cause the most rapid drop in
E
p
. This process is repeated in an iterative fashion
