COMPLEX-VALUED ADAPTIVE SIGNAL PROCESSING - Adaptive Signal Processing: Next Generation Solutions - page 55

such nonlinearities rather than the split type functions given in (1.43)-(1.45), the
update rules for the MLP can be derived in a manner very similar to the real case as
we demonstrate next for the derivation of the back-propagation algorithm.

1.5.2 Derivation of Back-Propagation Updates

For the MLP filter shown in Figure 1.17, we write the square error cost function as

J ( V , W ) ¼ X

K

( d k y k )( d k y k )

k¼ 1

where

!

y k ¼ h X

N

w kn x n

n¼ 1

and

!

x n ¼ g X

M
m¼ 1 v nm z m

:

When both activation functions h ( ) and g ( ) satisfy the property [ f ( z )] ¼ f ( z ), then
the cost function can be written as J ( V , W ) ¼ G ( z ) G ( z ) making it very practical to
evaluate the gradients using Wirtinger calculus by treating the two variables z and
z as independent in the computation of the derivatives. Any function f ( z ) that is ana-
lytic for jzj , R with a Taylor series expansion with all real coefficients in jzj , R
satisfies the property [ f ( z )] ¼ f ( z ) as noted in [6] and [71].
Examples of such functions include polynomials and most trigonometric functions
and their hyperbolic counterparts (all of the functions whose characteristics are shown
in Figs. 1.12-1.16), which also provide universal approximation ability as discussed
in Section 1.5.1. In addition, the activation functions given in (1.43)-(1.45) that pro-
cess the real and imaginary or the magnitude and phase of the signals separately also
satisfy this property. Hence, there is no real reason to evaluate the gradients through
separate real and imaginary part computations as traditionally done. Indeed, this
approach can easily get quite cumbersome as evidenced by [12, 14, 39, 41, 62, 67,
107, 108, 118] as well as a recent topic [75] where the development using
Wirtinger calculus is presented as an afterthought, with the result in [6] and [71]
that enables the use of Wirtinger calculus given without proper citation.
When the fully-complex functions introduced in Section 1.5.1 are used as acti-
vation functions as opposed to those given in (1.43)-(1.45), the MLP filter can
achieve significantly better performance in challenging signal processing problems
such as equalization of highly nonlinear channels [61, 62] both in terms of superior
convergence characteristics and better generalization abilities through the efficient