Neurocomputing with High Dimensional Parameters - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 31

existence of the derivative of a function at a point z . They can be used to locate points
at which function does not have a derivative. These equations are not sufficient to
ensure the existence of the derivative of a function at that point.
The complex plane unlike the real line is a two-dimensional space. The second
dimension adds flexibility and at the same time restricts the choice of activation
functions for neural network applications by imposing certain constraints. More
precisely, the important constraint imposed by the complex plane is epitomized in
the Liouville Theorem.

Theorem 2.1 The theory of functions in complex domain imposes its own constraint
in the form of Liouville Theorem, which states that if a function in complex domain
is both entire and bounded, it must be a constant function. As a ramification of the
theorem, the constraints emerge:

• No analytic function except a constant is bounded in the complex plane.

• An analytic complex function cannot be bounded on all points of the complex plane

unless it is constant.

Analyticity, boundedness are the parameters of concern in the search for complex
activation functions. The term regular and holomorphic are also interchangeably used
in the litrature to denote analyticity. In view of theorem, if a nontrivial complex-valued
function is analytic it must go unbounded at least one point on the complex plane,
and if the function is bounded it must be nonanalytic in some region for it to qualify
as activation function. Hence, a search for activation function should make sure these
conditions are satisfied. The second dimension of the complex plane necessitates a
study of three-dimensional surfaces (Chap. 3 ) , as the real and imaginary parts of the
complex activation functions are both functions of real and imaginary parts of the
variable.

Definition 2.5 If a function fails to be analytic at a point z 0 , but is analytic at some
point in every neighborhood of z 0 , then z 0 is called a singular point or singularity of
function.

2.3.2 Complex Variable Based Neural Networks

Complex numbers form a superset of real numbers, an algebraic structure that defines
real world phenomenon like signal magnitude and phase. These are useful in ana-
lyzing various mathematical and geometrical relationships in two dimension space.
For nearly a decade, the extension of real-valued neurons for operation on complex
signals [ 2 , 5 - 7 ] has received much attention in the field of neural networks. The
main motivation in designing complex variable based neural networks is to utilize
the promising capabilities of complex numbers. Complex numbers are a subfield of
quaternions. The decision boundary of the complex-valued neuron consists of two
hypersurfaces, which intersect orthogonally each other and divides a decision region