COMPLEX-VALUED ADAPTIVE SIGNAL PROCESSING - Adaptive Signal Processing: Next Generation Solutions - page 37

2 f
@ w C @ w C

@

where T W H 2 H 1 H 2 H 1 and ( ) denotes [( ) ] 1 . Since is Hermitian,

we finally obtain the complex Newton's method given in (1.31). The expression for
Dw is the conjugate of (1.31).

In [80], it has been shown that the Newton algorithm for N complex variables
cannot be written in a form similar to the real-valued case. However, as we have
shown, by including the conjugate of N variables, it can be written as shown in
(1.31), a form that is equivalent to the Newton method in R

2 n . This form is also
given in [110] using the variables w R and w C , which is shown to lead to the
form given in (1.31) using the same notation in [64]. Also, a quasi-Newton update
is given in [117] by setting the matrix H 1 to a zero matrix, which might not define a
descent direction for every case, as also noted in [64].

C N

1.3.3 Matrix Optimization in

Complex Matrix Gradient Gradient of a matrix-valued variable can also
be written similarly using Wirtinger calculus. For a real-differentiable f ( W , W ):
C

NN NN

C 7! R , we recall the first-order Taylor series expansion given in (1.20)

@f
@W

@f
@W

þ DW ,

Df DW ,

@f
@W

¼ 2Re DW , (1 : 33)

where @f

@W is an NN matrix whose ( m , n )th entry is the partial derivative of f with
respect to w mn . As in the vector case, the matrix gradient with respect to the conjugate
@f
@W defines the direction of the maximum rate of change in f with respect to the
variable W .

Complex Relative Gradient Updates We can use the first-order Taylor
series expansion to derive the relative gradient update rule [21] for complex matrix
variables, which is usually directly extended to the complex case without a derivation
[9, 18, 34]. To write the relative gradient rule, we consider an update of the parameter
matrix W in the invariant form G ( W ) W [21]. We then write the first-order Taylor
series expansion for the change of the form G ( W ) W as

@f
@W

þ G ( W ) W , @ f
@W

Df G ( W ) W ,

@f
@W W H

¼ 2Re G ( W ),