COMPLEX-VALUED ADAPTIVE SIGNAL PROCESSING - Adaptive Signal Processing: Next Generation Solutions - page 6

statistical characterization of a complex signal, which play an important role in the
subsequent discussions on widely linear filters and independent component analysis.
Next, we show how Wirtinger calculus enables derivation of effective algorithms
using two filter structures that have been shown to effectively use the complete statisti-
cal information in the complex signal and discuss the properties of these filters. These
are the widely linear and the fully complex nonlinear filters, two attractive solutions for
the next generation signal processing systems. Even though the widely linear filter is
introduced in 1995 [94], its importance in practice has not been noted until recently.
Similarly, the idea of fully complex nonlinear filters is not entirely new, but the theory
that justifies their use has been developed more recently [63], and both solutions hold
much promise for complex-valued signal processing. In Sections 1.4 and 1.5, we pre-
sent the basic theory of widely linear filters and nonlinear filters—in particular multi-
layer perceptrons—with fully complex activation functions using Wirtinger calculus.
Finally in Section 1.6, we show how Wirtinger calculus together with fully complex
nonlinear functions enables derivation of a unified framework for independent
component analysis, a statistical analysis tool that has found wide application in
many signal processing problems.

1.2 PRELIMINARIES

1.2.1 Notation
A complex number z [ C is written as z ¼ z r þ jz i where j ¼ p and z r and z i refer

to the real and imaginary parts. In our discussions, when concentrating on a single
variable, we use the notation without subscripts as in z ¼ x þ jy to keep the
expressions simple. The complex conjugate is written as z ¼ z r jz i , and vectors
are always assumed to be column vectors, hence z [ C

N 1 .
In Table 1.1 we show the six types of derivatives of interest that result in matrix
forms along with our convention for the form of the resulting expression depending
on whether the vector / matrix is in the numerator or the denominator. Our discussions
in the chapter will mostly focus on the derivatives given on the top row of the table,
that is, functions that are scalar valued. The extension to the other three cases given in

N implies z [ C

Table 1.1 Functions of interest and their derivatives

Vector Variable:
z [ C

Matrix Variable:
Z [ C

Scalar Variable:
z [ C

N NM

[ C [ C

@f
@z [ C

Scalar Function:
f [ C

@f

@z ¼ @ f @Z ¼ @ f

@f

N NM

@z k @Z kl

[ C

@ f
@z [ C

Vector Function:
f [ C

@ f

@z ¼ @ f l

1 L NL

L

@z k

@ F
@z [ C

Matrix Function:
F [ C

KL

LK