Application of Artificial Neural Networks - Digital Signal Processing in Power System Protection and Control - page 250

neuron
synapse

neuron

layer layer

output

input
layer

hidden
layer ( -s)

output
layer

Fig. 12.8 Multilayer perceptron structure

Fig. 12.9 Selected non-
linear neuron activation
functions

Wp ? b at the neuron output. Instead of the hard limit (binary, threshold) a
continuous, smooth, differentiable function is introduced, with the output limited
to the range [0; 1] or [-1; 1], see Fig. 12.9 .
The two most popular differentiable activation functions are [ 9 ]:

• Sigmoidal logistic function

1
1 þ exp ð b e Þ

y ¼ ð 12 : 3 Þ