Information Technology Reference
In-Depth Information
The foreword and the reading guide will help the reader to navigate through
the chapters as a function of his own topics of interest.
1.6 Additional Material
1.6.1 Some Usual Neurons
Two categories of neurons can be distinguished, depending on the role of their
parameters.
1.6.1.1 Neurons with Parameterized Inputs
The most popular neurons are neurons with parameterized inputs, in which
one parameter is assigned to each input. The output of a neuron having n
inputs {x i } , i =0to n − 1, is therefore given by an equation of the form
y = f
{
x i ,w i }
, i =0to n
1, where
{
w i }
, i =0to n
1 are the parameters
of the model.
In most cases, function f is the composition of two operations,
the computation of the potential v of the neuron, which is the sum of the
inputs of the neuron, weighted by the corresponding parameters,
v = n− 1
w i x i ;
i =0
the computation of a nonlinear function of the potential, termed activation
function; that function is generally s -shaped, hence the generic name of
sigmoid; preferably the activation function is symmetric with respect to
the origin, such as the tanh function or the inverse tangent function, except
if some prior knowledge on the problem prompts the implementation of
different, more appropriate functions.
The set of inputs of the neuron generally includes a specific input, termed bias,
the value of which is constant, equal to 1. It is usually assigned the index 0,
so that the potential is of the form
v = w 0 + n− 1
w j x j .
j =1
Thus, the expression of the output of the neuron is: y = f [ w 0 + n− 1
j =1 w j x j ].
Figure 1.50 shows the output of a neuron with three inputs ( x 0 =1, x 1 ,
x 2 ) with parameters w 0 =0, w 1 =1, w 2 =
1.
Search WWH ::




Custom Search