Information Technology Reference
In-Depth Information
be improved substantially. The statement was substantiated by demonstrating that an
Absolute EF-based optimization solved a curve-fitting problemmore efficiently than
the standard Quadratic EF-based optimization. The observation was attributed to the
fact that the cost accrued is greater if a power term exists; the outlier points hence
distort the optimization scheme. On the other hand, an Absolute EF is equi-poised
toward data points and the outliers and therefore the ill-effects of the outliers are
more balanced in the scheme resulting in a better curve-fit.
Earlier Werbos and Titus (1978), Gill and Wright (1981) also discussed the idea
of changing the EF in an optimization scheme. Fernandez (1991) implemented some
new EFs that were designed to counter the ill-effects local minima by weighting
the errors according to their magnitudes. Matsuoka (1991) reported BPA based on
Logarithmic EF and elimination of local minima. Ooyen and Nienhaus (1992) used
an entropy type EF and showed that it performs better than the Quadratic EF-based
BP for function approximation problems. It is worth to mention here that the data set
obtained while practically experimenting is prone to system noise, process noise, and
measurement errors (like parallax). The outlier points contribute to the offset in the
solution to the curve-fitting problem (Rey 1983). There exist two approaches to tackle
the undesirable affects of spurious data points. The first approach demands that these
points be eliminated completely (by some data processing technique) and later after
weeding out these points, subject the data to a Quadratic EF-based optimization
scheme and obtain a solution, which can be termed ideal approach. The second
approach, as explained in Rey (1983) requires incorporating a modified EF that
would by the nature of design and construction have useful properties to bypass the
ill-effects of the spurious points in the data and obtain a better fit of curve to the data
set than the Quadratic EF. Therefore, there is a requirement to identify the properties
that useful EFs in real and complex domain must satisfy and subsequently investigate
learning rules so that user can overcome from the drawbacks of different functions
and select for better optimization. The next section presents various important EFs
in real and complex domain with viewpoint of backpropagation learning algorithm
in corresponding real (BP) and complex ( C BP) domain.
3.3.2 Definitions and Plots of Error Functions
This section presents few prominent EFs in view of neural network learning proce-
dure, though these EFs were proposed in literature to implement an m-estimators
approach for bypassing or reducing the ill-effects of the outliers. The functions in
each case have been generalized in a way as to retain the form and yet be operational
in the complex variable setting. This also makes sure that the surface plot of the func-
tion is close to the plane plot of the same. The definition EFs in real and complex
domain and their corresponding plane and surface graphs displaying their overall
shape are demonstrated. It could be appreciated from the graphs that retaining the
form of the function manifests itself in the plots as they resemble their counterparts
on the plane. Let e
R and
ʵ
C be the error and E be the EF for optimization in
 
Search WWH ::




Custom Search