Neurocomputing in Complex Domain - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 76

3.5 Concluding Remarks
The approach in the chapter went along the following lines: The two important para-
meters that exist in the neural network learning viz Activation Function and Error
Function were thoroughly discussed with associated properties. The chapter is fur-
ther extended to an improved and fast leaning algorithmwhich counters the ill effects
of popular back-propagation learning algorithm in complex domain. Of the various
parameters set to run the CVNN, the C AF is most critical. In the complex domain,
analyticity of the functions is supposed to be verified before it can be used as an acti-
vation function to the CVNN. This aspect is unlike real-valued ANN where no such
constraint existed and AF chosen to be a smooth (continuously differentiable) and
bounded function. On the other hand, the complex plane imposes its own constraint
in the form of Liouville Theorem, which states that if a complex-valued function is
both analytic and bounded, it must be a constant function. It is useless to consider
a constant function for neuron's activation to accomplish learning process therefore
many researchers have favored for the boundedness of C AF . The Liouville Theorem
constraint is studied here and the different C AFs proposed are surveyed. A study into
the boundedness behavior of the so-called “split” type function for activation of neu-
ron is revealed in cost of its analytic behavior. The broad picture of C AFs available
from literature have been collected and analyzed. The C AFs, Haykin Activation and
Split Activation were comparatively explained. The singular points of the Haykin
fully complex-valued activation function, which were found to be responsible for
disrupting the downstream convergence whenever at least one of the net inputs to
neurons fell in their vicinity in the course of training. In 2003, Adeli [ 11 ] thoroughly
explored fully complex-valued function for CVNN and stated that the universality
of the networks can be shown by dealing with these singularities, such that singu-
larities are removed or avoided by restricting the regions. The split-type function for
activation found to be a better choice for construction of CVNN in all respects.
This chapter presents prominent EFs as a basis for making these choices and
designing a learning scheme. This approach will clearly offer more tooling while
designing neural networks as firstly, it departs from the present day technique of
using the quadratic EF and invoking the procedure of weight update using a gradient
descent, which in practice does not manage to go below a certain value of the error
(although theory assures that an neural network exists for an arbitrary error that the
investigator may desire). The study reveals that the EF can indeed be treated as a para-
meter while employing learning in RVNN or CVNN for training of presented data.
The deployment of other EFs that may perform well and can replace the quadratic
EF is contingent upon the applications or requirements. It must be noted that in the
description of EFs and the function's form has been retained while extending to the
complex domain. This was done to make sure that the error computed kept the same
formula and also makes sure that the surface plot of the function is close to the plane
plot of the same, even while operating in the complex domain.
The main drawback of BP is the large computational complexity due to extensive
training. One way to decrease the computational load in multilayer network is to