Neurocomputing in Complex Domain - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 47

Table 3.1 Complexity of the neural network with complex domain neuron and real domain neuron
for C XOR Problem
Network

Time complexity Space complexity

Based on × and ÷ + and − Sum Weights Thresholds Sum

Complex domain neuron 134 92 226 24 10 34

2-4-1
Real domain neuron 150 76 226 54 11 65

4-9-2

3.2 Activation Functions in Complex Domain

There were a spectrum of functions that performed activation to a real variable-based
neuron of the ANN. In fact, pieces of well-known curves could be joined to tailor
curves that looked like the sigmoid (and possessed derivatives to the required order)
of the ANN, which when applied as activation functions did obtain a convergent
sequence of error with epochs. On the other hand heresy to the flexibility in the choice
of activation functions in the ANN, the CVNN is constrained by some additional
facts that are embedded into the complex domain in which it operates. The Liouville
Theorem [ 15 , 16 ] imposes more constraint on the fully complex valued functions.
The constraint imposed by the theorem is additional in the complex variable setting
as an equivalent did not exist in the real variable based ANN. Apart form fully
complex-valued activation functions, a wide variety of activation functions have been
investigated including the split-type function [ 9 , 17 , 18 ], phase-preserving function
[ 7 ] and circular-type function [ 19 ].
Complex Activation Functions ( C AF) and their characteristics depart from the
traditional activation functions of the ANN in real domain. First, these functions have
real and imaginary parts each of which individually are functions of two variables
that make them surfaces in three-dimensional space, the analyticity of which plays
a vital role of course. Second, there exist additional constraints imposed by the
complex plane in the form of Liouville Theorem (Ahlfors 1979) that restrict the
choice of functions that could be used as the C AF. As a result, tailoring new C AF by
sewing pieces of surfaces along the common boundary would not be an acceptable
proposition as the analyticity of the function developed this way should be established
at each point on the boundary and later the construct must be verified to have cleared
the constraint imposed by the Liouville Theorem. This restriction is unlike real value
activation functions that could be easily tailored by joining differentiable functions
and establishing differentiability at finitely many points (where each piece joins
up with the next). In this chapter, two main broad directions of C AFs reported by
researchers have been investigated in the above context.

Theorem 3.1 Liouville Theorem states that 'If a complex valued function is both
analytic and bounded through out the complex plane, then it must be a constant