Higher-Order Computational Model for Novel Neurons - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 83

Fig. 4.1 An artificial
neuron model with
aggregation
function ʩ
use of neural networks with conventional neurons is naturally accompanied by the
trade-off between issues, such as the complexity of network, overfitting, generaliza-
tion capability, local minima problems, and stability of the weight update system.
In recent past, much attention has been given to convergence and stability evalua-
tion of neural network weights by energy function-based optimization methods, to
training techniques to avoid weights from getting stuck into local minima and to
avoid overfitting. Mostly, those techniques are sophisticated; and require substantial
and time-consuming (hence costly) effort from users who are not true experts in the
neural network field.
In order to overcome with these issues, one reasonable approach is to be concerned
with neuron modeling which relates the structure of a neuron with its aggregation
function. Neurons are functional units and can be considered as generators of func-
tion spaces over impinging signals. The major issue in artificial neuron model is the
description of signal aggregation which reveals the power of single neuron compu-
tation. It was observed in many researches [ 5 , 7 , 8 ] that the functional capability of a
neuron lies in the spatial integration of synaptic inputs. Probably not much attempts
have been made for the evolution of more powerful neuron models to enhance the
overall performance and computational capabilities of ANN. A neuron designed
with higher-order correlation over impinging signals leads to a superior mapping
and computing capabilities [ 9 ]. Therefore, multiplication units [ 7 ] have became a
natural choice in modeling a computationally powerful and biologically plausible
extension to conventional neurons. Further, the most well-known units that com-
prise of multiplicative synapses are perhaps higher-order neurons; which have been
developed to enhance the nonlinear expressional ability of the multilayer neural net-
works. Motivated by the higher-order characteristics of the neuron and the classic
Stone-Weierstrass's theorem [ 10 ], a class of neuron models known as pi-sigma [ 11 ],
sigma-pi [ 12 ], polynomial neural networks [ 13 ], and higher-order neurons [ 9 ]have
been introduced and successfully used. These models have been proved to be more
efficient as both single units as well as in networks. These neuron models forms the
higher-order polynomials on the basis of the number of inputs in space, improves
the learning capability in terms of speed and performance with lesser number of