MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 70

because, as seen before, in the divergence phase the fluctuations decrease to zero.
The other algorithms do not have this feature; they need low initial conditions
( which implies high variance ) and have low bias, but with high oscillations around
the solution for a longer time than MCA EXIN.

2.7.2 OJA

The analysis on OJA can be found in [195, proof of Prop. 2]. There is dynamic
instability (

ρ > 1) when

1
2 p

cos 2

ϑ x w < ∧ α > α b

(2.160)

being

2

α b =

2 1 − 2 p cos 2

ϑ x w

(2.161)

x ( t )

The first condition implies the absence of the negative instability (i.e., > 0).

In reality, the first condition is contained in the second condition. Indeed, con-
sidering the case 0 <α b ≤ γ< 1, it holds that

α b

1
2 p −

1

cos 2

ϑ x w ≤

2 = ϒ

(2.162)

2

γ p x ( t )

which is more restrictive than (2.178). Figure 2.13 shows this condition, where

σ = arccos √ ϒ . The instability domain is the contrary of the domain of the

neurons considered previously. The angle σ is proportional to 1 / √ 2 p and then
an increasing weight modulus (as close to the MC direction for λ n > 1) better
respects this condition. Also, the decrease of γ has a similar positive effect. From
Figure 2.13 it is apparent that in the transient (in general, low ϑ x w ), there are
fewer fluctuations than in the neurons cited previously. Then OJA has a low
variance but a higher bias than MCA EXIN [see eq. (2.139)].

2.7.3 FENG

From eq. (2.29) it follows that

T

w ( t + 1 ) x ( t ) = y ( t )( 1 − α q ) (2.163)

where

2
2

q = p x ( t ) − 1 (2.164)

and

p 1 )

2
2

2

w ( t + 1 ) = − 2 α ( u − 1 ) + α ( qu − u + 1 (2.165)