Information Technology Reference
In-Depth Information
In this section, MMSE-batch and MEE-based learning algorithms for the
single layer CVNN, with the architecture proposed in [9], are presented.
6.3.2 Single Layer Complex-Valued NN
This subsection follows closely [9]. Consider an input space with
d
features.
The neuron input is the complex vector
x
=
x
R
+
ix
I
where
x
R
is the real
and
x
I
d
canalsobewrittenas
w
=
w
R
+
iw
I
. The net input of neuron
k
is given by
d
. The weight matrix
w
is the imaginary part, such that
x
∈
C
∈
C
d
z
k
=
θ
k
+
w
kj
x
j
,
(6.31)
j
=1
where
θ
k
∈
C
is the bias for neuron
k
and can be written as
θ
k
=
θ
k
+
iθ
k
.
Given the complex multiplication of
w
k
x
, this can further be written as
+
iz
k
=
θ
k
x
I
w
k
+
i
θ
k
+
x
R
w
k
+
x
I
w
k
.
z
k
=
z
k
+
x
R
w
k
−
(6.32)
Note that,
m
x
R
w
k
=
x
j
w
kj
.
(6.33)
j
=1
The
k
neuron output is given by
y
k
=
f
(
z
k
) where
f
:
C
→
R
is the activation
. The activation function used is
f
(
z
k
)=(
s
(
z
k
)
s
(
z
k
))
2
function and
y
k
∈
R
−
1
1+exp(
where
s
(
x
)
.Giventheformofthis
activation function, it is possible to solve non-linear classification problems,
whereas in the case of a real-valued neural network with only one layer (such
as a simple perceptron) this would not be possible. Now that the output
of the single complex-valued neuron is defined, the way to train a network
composed of a single layer with
N
of these complex-valued neurons can be
presented. The network is trained in [9] by applying gradient descent to the
MMSE functional
·
) is the sigmoid function
s
(
x
)=
−
N
E
(
w
)=
1
2
y
k
)
2
,
(
t
k
−
(6.34)
k
=1
where
t
k
∈
R
represents the target output for neuron
k
. To minimize (6.34)
the derivative w.r.t. the weights is used:
s
(
z
k
))
s
(
z
k
)
x
j
−
s
(
z
k
)
x
j
.
∂E
∂w
kj
2
e
k
(
s
(
z
k
)
=
−
−
(6.35)
The previous expression is the derivative w.r.t. the real weights but a similar
one should be made w.r.t. the imaginary weights. To obtain the weights, the
