Complex differential calculus (Wirtinger calculus) - Statistical Signal Processing of Complex-Valued Data - page 277

Appendix 2 Complex differential
calculus (Wirtinger calculus)
In statistical signal processing, we often deal with a real nonnegative cost function, such
as a likelihood function or a quadratic form, which is then either analytically or numer-
ically optimized with respect to a vector or matrix of parameters. This involves taking
derivatives with respect to vectors or matrices, leading to gradient vectors and Jaco-
bian and Hessian matrices. What happens when the parameters are complex-valued?
That is, how do we differentiate a real-valued function with respect to a complex
argument?
What makes this situation confusing is that classical complex analysis tells us that a
complex function is differentiable on its entire domain if and only if it is holomorphic
(which is a synonym for complex analytic ). A holomorphic function with nonzero
derivative is conformal because it preserves angles (including their orientations) and the
shapes of infinitesimally small figures (but not necessarily their size) in the complex
plane. Since nonconstant real-valued functions defined on the complex domain cannot
be holomorphic, their classical complex derivatives do not exist.
We can, of course, regard a function f defined on C n as a function defined on IR 2 n .If
f is differentiable on IR 2 n ,itissaidtobe real-differentiable , and if f is differentiable
on C n ,itis complex-differentiable . A function is complex-differentiable if and only
if it is real-differentiable and the Cauchy-Riemann equations hold. Is there a way to
define generalized complex derivatives for functions that are real-differentiable but not
complex-differentiable? This would extend complex differential calculus in a way similar
to the way that impropriety extends the theory of complex random variables.
It is indeed possible to do this. The theory was developed by the Austrian mathe-
matician Wilhelm Wirtinger (1927 ), which is why this generalized complex differential
calculus is sometimes referred to as Wirtinger calculus . In the engineering literature,
Wirtinger calculus was rediscovered by Brandwood (1983 ) and then further developed
by van den Bos (1994a ). In this appendix, we mainly follow the outline by van den Bos
(1994a ), with some minor extensions.
The key idea of Wirtinger calculus is to formally regard f as a function of two
independent complex variables x and x ∗ . A generalized complex derivative is then
formally defined as the derivative with respect to x , while treating x ∗ as a constant.
Another generalized derivative is defined as the derivative with respect to x ∗ , while
formally treating x as a constant. The generalized derivatives exist whenever f is real-
differentiable. These ideas extend in a straightforward fashion to complex gradients,
Jacobians, and Hessians.