Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 6

Fig. 1.3. A neural network with n + 1 inputs, a layer of N c hidden neurons with
sigmoid activation function, and a linear output neuron. Its output g ( x , w )isa
nonlinear function of the input vector x , whose components are 1 ,x 1 ,x 2 ,...,x n ,
and of the vector of parameters w , whose components are the ( n +1) N c + N c +1
parameters of the network

of the parameters of the first layer of connections (connections that convey
information from the n + 1 inputs of the network to the N c hidden neurons).
That property has important consequences, which will be described in detail
in a subsequent section.
The output of a multilayer perceptron is a nonlinear function of its inputs
and of its parameters.

1.1.1.2 What Is a Neural Network with Zero Hidden Neurons?

A feedforward neural network with zero hidden neuron and a linear output
neuron is an a ne function of its inputs. Hence, any linear system can be re-
garded as a neural network. That statement, however, does not bring anything
new or useful to the well-developed theory of linear systems.

1.1.1.3 Direct Terms

If the function to be computed by the feedforward neural network is thought
to have a significant linear component, it may be useful to add linear terms
(sometimes called direct terms) to the above structure; they appear as addi-
tional connections on the graph representation of the network, which convey
information directly from the inputs to the output neuron (Fig. 1.4). For
instance, the output of a feedforward neural network with a single layer of
activation functions and a linear output function becomes

⎡ ⎛ ⎞ ⎤

g ( x , w )= N c

n n

⎣ w N c +1 ,i tanh ⎝ ⎠ ⎦ +

w ij x j w N c +1 ,j x j .

i =1 j =0 j =0