Knowledge Discovery and Data Mining in Medicine - Advanced Techniques in Knowledge Discovery and Data Mining - page 181

Fig.7.4. Convergence of the input weight of a neuron.

representation power to learn the subproblem, the neuron was split into two, i.e.,
we added another neuron to the same interconnection of the network as its parent
neurons' attributes are inherited.
Figure 7.4 shows an idea for converging an input weight of a neuron through
searching optimal spaces of weights W , W .
During the convergence process, the average length of movement between
adjacent temporal weights is gradually decreased. On the basis of this observation,
we define the following measure, called the walking distance of the

( WD )

neuron:

& &

n

[] [] [ ]

∑ =

WD n = Met ( W m , W m − 1 )

(7.1)

j j j

m

[] where WD j n is the WD of neuron j in the hidden layer at iteration n ,

&

[]

is the input weight of neuron j at arbitrary iteration m , and Met is a
metric that measures the distance between vectors in a metric space. In this
chapter, Met is the Euclidean metric:
( )

W j m

& &

N

( )

2

EuMet X , Y =

∑ −

x y

i i

i = 1

, (7.2)

where

& &

N = Dim ( X ) or Dim ( Y )

. (7.3)

In Eq. (7.1), WD is a measure for the time variance of the stored information

for neuron j ; it can be considered the activity of neuron j in the parameter
space. Too large a WD of a particular neuron indicates that the processing
capability of the neuron is insufficient and that the load should be shared with

j