Higher-Order Computational Model for Novel Neurons - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 85

sive attempt has been by Shin and Ghosh (1991) [ 11 ], Chen and Billings (1992)
[ 15 ], Chen and Manry (1993) [ 13 ], Schmidt and Davis (1993) [ 16 ], Kosmatopoulos
et al. (1995) [ 17 ], Heywood and Noakes (1996) [ 12 ], Liu et al. (1998) [ 18 ], Elder and
Brown (2000) [ 19 ], Bukovsky et al. (2003) [ 20 ], Hou et al. (2007) [ 21 ] and Bukovsky
et al. (2009) [ 8 ] to develop the foundation of nonconventional neural units.
Motivated by their efforts, a class of structures known as pi-sigma and sigma-pi
[ 11 , 12 ], functional link networks [ 15 ], polynomial neural networks [ 13 , 18 , 19 ],
higher-order neuron [ 16 , 17 , 21 ], quadratic, and cubic neural units [ 20 ] have been
introduced. Significant and most recent publications devoted to polynomial neural
networks concepts are the works of Nikolaev and Iba [ 22 ] while most recent works
that are framed within higher-order neuron can be found in [ 8 , 23 ]. Although, higher-
order neurons have proved to be most efficient, but they suffer from the typical curse
of dimensionality due to combinatorial explosion of terms when there is increase in
the number of inputs. In fact, with the increase of the input dimension, the number
of parameters that is weights, increases rapidly and becomes unacceptably large
for use in many situations. Consequently, typically only small orders of polynomial
terms are often considered in practice. This problem worsens when higher-order
neurons are implemented in complex domain. These studies inspired to design new
neuron models, which capture nonlinear correlation among input components but are
free from the problem of combinatorial explosion of terms as the number of inputs
increases. Therefore, in this chapter three new higher-order artificial neuron models
have been presented with solid theoretical foundation for their functional mapping.
4.3 Novel Higher-Order Neuron Models
Neurons are functional units and can be considered as generators of function spaces.
The artificial models of neuron are characterized by their formalism and their precise
mathematical definition. Starting from linear aggregation proposed by McCulloch-
Pitts model [ 4 ] in 1943 and Rosenblatt perceptron [ 24 ] in 1958 to higher-order
nonlinear aggregations [ 8 , 9 , 19 ], a variety of architectures of the neurons have been
proposed in literature. It has beenwidely accepted that the computational power of the
neuron depends on the structure of aggregation function. Higher-order neurons have
demonstrated improved computational power and generalization ability. However,
these units are difficult to train because of a combinatorial explosion of higher-order
terms when the number of inputs to the neuron increases. This section presents three
higher-order neuron models which have a simpler structure without above issue;
hence need not to bother for selecting the relevant monomials or the requirement of
sparseness in representation that was necessary to be imposed on the other higher-
order neurons to keep learning practical. These models can also be used in the same
form in networks of similar units or in combination with the traditional neuron
models.
Another direction is related to the domain of implementation. It is worth men-
tioning here that the performance of neurons in complex domain is far superior for