COMPLEX-VALUED ADAPTIVE SIGNAL PROCESSING - Adaptive Signal Processing: Next Generation Solutions - page 48

the real domain for problems that require nonlinear signal processing capability [42].
Both the MLP and the RBF filters are shown to be universal approximators of any
smooth nonlinear mapping [30, 35, 51] and their use has been extended to the complex
domain, see for example [12, 14, 67, 108].
A main issue in the implementation of nonlinear filters in the complex domain has
been the choice of the activation function. Primarily due to stability considerations,
the importance of boundedness has been emphasized, and identified as a property
an activation function should satisfy for use in a complex MLP [36, 119]. Thus, the
typical practice has been the use of split-type activation functions, which are defined
in Section 1.2.1. Fully-complex activation functions, as we discuss next, are more
efficient in approximating nonlinear functions, and can be shown to be universal
approximators as well. In addition, when a fully-complex nonlinear function is used
as the activation function, it enables the use of Wirtinger calculus so that derivations
for the learning rules for the MLP filter can be carried out in a manner very similar to
the real-valued case, making many efficient learning procedures developed for the
real-valued case readily accessible in the complex domain. These results can be
extended to RBF filters in a similar manner.

1.5.1 Choice of Activation Function for the MLP Filter

As noted in Section 1.2.2, Liouville's theorem states the conflict between the bound-
edness and differentiability of functions in the complex domain. For example, the tanh
nonlinearity that has been the most typically used activation function for real-valued
MLPs, has periodic singular points as shown in Figure 1.13.
Since boundedness is deemed as important for the stability of algorithms, a prac-
tical solution when designing MLP filters for the complex domain has been to
define nonlinear functions that process the real and imaginary parts separately through
bounded real-valued nonlinearities as defined in Section 1.2.1 and given for the
typically employed function tanh as

f ( z ) W tanh( x ) þ j tanh( y ) (1 : 43)

for a complex variable z ¼ xþ jy where tanh: R 7! R . The activation function can
also be defined through real-valued functions defined for the magnitude and phase
of z as introduced in [45]

m e ju

r

f ( z ) ¼ f ( re ju ) W tanh

(1 : 44)

where m is any number different than 0. Another such activation function is proposed
in [36]

z
cþjzj=d

f ( z ) W (1 : 45)

where again c and d are arbitrary constants with d = 0. The characteristics of the acti-
vation functions given in (1.43)-(1.45) are shown in Figures 1.9-1.11. As observed