Discrimination - Neural Networks: Methodology and Applications - page 330

• It may be essential to define a class of rejected patterns (that the classifier

is unable to discriminate).

As in the previous chapters, we will consider problems where the data are
represented by vectors. Their components are characteristics that are relevant
to the discrimination task. For example, in the case of medical diagnosis, these
are the patient age, blood pressure, etc.; in the case of pattern recognition,
the pixels of the image ...The classes may be encoded as integer numbers,
representing the kind of disease, or the image type. We will mainly consider
problems where the data can only belong to one of two classes. As will be
discussed in the corresponding section, problems with more than two classes
may be reduced to a set of two-class problems.
The chapter is divided into five sections. After general considerations, we
describe several training algorithms for linear separation. Then, we present
various cases where the discriminant surfaces are more complex. In the fourth
section, we consider the discrimination in problems with more than two
classes. At the end of the chapter we describe theoretical concepts, such as
the Vapnik-Chervonenkis dimension and the capacity of a classifier, which are
important in applications.

6.1 Training for Pattern Discrimination

Can we learn to classify new patterns using the information contained in a set
of examples previously classified by an expert? This is a variant of the general
problem already considered in previous chapters, where we tried to predict the
behavior of a process on fresh data, not used to adjust the model's parameters.
As explained in Chap. 1, regression and discrimination are ill-posed problems.

Remark. Some authors use the term “discrimination” to refer to the clas-
sification task when the classes are known a priori. That is the case of the
so-called supervised learning , in contrast to non-supervised learning, whose
goal is to organize data not previously classified. In this chapter, we con-
sider supervised learning of classification tasks, that we will loosely call either
classification or discrimination.

As in other training problems, the parameters of the classifier are estimated
from a training set of M examples L M , where each example is an input vector
and its class,

( x 1 ,y 1 ) , ( x 2 ,y 2 ) ,..., ( x M ,y M )

L M = { } ,

where the input

x k =[ x 1 ,x 2 ,...,x k M ] T
is a vector of N discrete, binary or real-valued components, describing the
example k ( k =1 , 2 ,...,M )and y k ∈{−

1 , +1 } is its class.