Neural Networks Approach - Computational Intelligence in Time Series Forecasting - page 77

x dendrites that feed the external signals to the cell body
x axons that carry the signals out of the cell to other cell bodies
This configuration was translated in terms of analogue computational technology
as shown in Figure 3.1, where

x the core part of the element, called a perceptron, contains a summing
element Ȉ and a nonlinear element NL
x the multiple signal inputs x are connected via adjustable weighting
elements w with the core part of the element
x the signal output(s) y

d

An additional perceptron input

0 w called the bias , is understood as a threshold

(switching) element.

Inputs weights X 0 = 1 bias output

Dendrites

w 1

w 0

x 1

Axon

w 2

Soma y 0

Nonlinear
Element

Summing
Element

x 2

:
: Neuron w n

Perceptron

x n

Figure 3.1. Symbolic representation of neuron and perceptron

The output signal is defined as

n

yf wxw

¦

0 ii 0

i 1

and the bias follows the relationship

wx w

T

t 0

0

meaning that the perceptron fires , i.e. it is activated and produces an output signal
when this condition is met, otherwise not.
Our attention should now be shifted to the question of what nonlinear function
should be implemented in the core part of the perceptron as its activation function .
The early attempt of Block (1962) to select the binary step function for this
purpose was later modified in favour of a sigmoid activation function (Figure 3.2).

1

fx ()

.

1 exp( x )