Parameterizing neural network models to improve land classification performance - Urban Remote Sensing: Monitoring, Synthesis and Modeling in the Urban Environment - page 99

used the confusion matrix method described earlier to measure
the classification accuracy. A test dataset with approximately 50
samples randomly selected for each land use/cover subclass/class
was used for this purpose. The overall classification accuracies
were compared to evaluate the performance of each training
algorithm.
Based on Table 7.6, it is clear that the classification accuracy
varies by training algorithms. If using the average classification
accuracy as the reference, the SGD and LM algorithms clearly
performed the best, followed by the GMD, RP, CGP, and SCG
algorithms. The BFG, CGB, and CGF algorithms generated rela-
tively lower average classification accuracies, which were mostly
caused by their failure to converge in one or more experiments.

7.5 Toward a systematic
approach to image
classification by neural
networks

While the prospect of neural networks for image classification
has been quite promising, the capability of neural networks
tends to be oversold as an all-inclusive 'panacea' that is capable
to outperform other classifiers regardless of network architec-
ture, training algorithms, or data quality. Consequently, this
field has been characterized by inconsistent research designs
and immature operational practices. Nevertheless, recent studies
suggested the need to adopt a systematic approach to pattern
recognition by neural networks considering data acquisition and
pre-processing, network configuration, training algorithms, and
validation in a sequential mode (e.g., Maier and Dandy, 2000;
Principe, Euliano and Lefebvre, 2000; Mas and Flores, 2008;
Yang, 2009a, b).
Here, we propose a systematic approach that can guide the use
of neural networks for image classification from remote sensor
data (Fig. 7.4). It comprises several core technical components,
beginning with data collection and acquisition that have been
considered as a critical component in any remote sensing-based
land use/cover mapping projects (Yang and Lo, 2002). At this
stage, both remote sensor data and ancillary data should be
collected to help prepare a land use/cover classification scheme,
the training and validation data sets, and the primary data
actually used in image classification. Every effort should be made
to acquire sufficient data that represent the conditions that the
neural network may encounter later. The information contents
and data quality should be emphasized during the phase of data
acquisition (Yang, 2009a).

7.4.3.4 Stability of the classification
accuracy

The stability of the classification accuracy was another criterion
that we used to evaluate the performance of a training algo-
rithm. The training methods which result in relatively stable
classification accuracies are generally preferred. To evaluate the
stability of the classification accuracy, the standard deviation of
the classification accuracies from the ten experiments for each
algorithm was computed. A smaller standard deviation suggests
a more stable performance. Based on Table 7.6, it is found that
the three back-propagation algorithms, namely, SGD, GDM,
and RP, and the LM algorithm were quite stable, the CGP and
SCG algorithms were less stable, and the CGF, CGB and BFG
algorithms were least stable. The poor performance of the last
group of training algorithms was clearly attributed to their poor
capability of convergence.

7.4.3.5 Overall evaluation

When combining the four categories of comparison, it is found
that the LMalgorithmperformed the best. It was themost efficient
algorithm with a strong capability of convergence, providing the
most accurate and stable land use/cover classification accuracies.
However, this algorithm may become less efficient when a large
memory space is needed to accommodate the high complexity
of computation due to increasing network size. The RP algo-
rithm performed almost identical to the LM algorithm except
that the former used more iterations to converge. Since the RP
algorithm does not require computing the matrix of the second-
order derivatives of its performance function, the networks size
increase will have a limited impact upon the performance. With
a much lower computational cost and less memory usage, the RP
algorithm can outperform the LM algorithm when training large
networks with many parameters. The CGP and SCG algorithms
also performed well although they showed moderate training
efficiency and the resultant classification accuracies were not as
stable as those from the LM and RP algorithms. The SGD and
GDM algorithms were not competitive when compared to the
LM, RP, CGP, and SCG algorithms due to their extremely low
training efficiency. Finally, the CGF, CGB, and BFG algorithms
were not recommended as they showed a poor capability of con-
vergence which led to poor classification accuracies. The above
observations suggest the importance of selecting an appropriate
training algorithmwhen using neural networks for land use/cover
classification.

Neural Network
Training

Neural Network
Configuration

Data
Preprocessing

Image
Classification

Evaluation
and Validation

Data Collection
and Acquisition

FIGURE 7.4 A systematic approach to image classifi-
cation by neural networks. While data acquisition and
pre-processing are normally quite time consuming, neu-
ral network configuration is most challenging. Nevertheless,
neural network training can be quite computationally inten-
sive.