Information Technology Reference
In-Depth Information
4.3.2 Feature Extraction Based on Decision Boundary
Another family of FE techniques is based on the properties of decision border [
10
].
Classes are statistically characterized by the
class-conditional probability density
function
(cpdf)
p
X
|
Y
(
x
|
y
i
)
, where the continuous random vector
X
takes values in
N
and the discrete random variable
Y
takes value in
y
. The cumulative probability
density function of the random vector
X
is:
R
C
p
X
(
x
)
=
P
Y
(
y
i
)
p
X
|
Y
(
x
|
y
i
),
(4.1)
i
=
1
where
P
Y
(
is the a-priori probability of class
y
i
.
Therefore, a classification or decision rule is a mapping
y
i
)
N
Y
that assigns
a class label to data on the basis of the observation of its feature vector. Aclassification
rule determines a partition of the feature space in
C decision regions D
1
,...,
ʨ
:
R
ₒ
D
C
N
such that
D
i
={
. The boundary separating decision regions is
called the
decision boundary
. Figure
4.1
illustrates an example of decision rule for
two Gaussian classes (symbolized by '
x
∈
R
|
ʨ(
x
)
=
y
i
}
' and 'o'). The straight line represents the
decision boundary: all points at the left of it are assigned by the decision rule to '
∗
∗
'
class, and those at the right to 'o' class.
Among all possible classification rules, the rule achieving the minimum
error
probability
ʵ
=
(
|
y
i
)
(
y
i
)
p
x
P
d
x
(4.2)
y
i
=
ʨ(
x
)
is the Bayes rule
The corresponding decision
boundary is consistently called
Bayes boundary
, which is the theoretically optimal
solution that every classification method aims to achieve.
The geometry of the decision boundary has been used in the discriminative feature
extraction approach known as
Decision Boundary Feature Extraction
(DBFE) [
25
]
ʨ
B
(
x
)
=
arg MAX
y
i
[
p
(
x
|
y
i
)
P
(
y
i
)
]
.
(a)
(b)
4
5
3
4
2
(c)
3
4
1
3.5
2
0
3
2.5
−1
1
2
−2
1.5
0
1
−3
0.5
-1
0
−4
−4 −3 −2 −1 0 1 2 3 4
0
1
2
3
4
5
6
7
8
9
-2 -1
0
1
2
3
4
Fig. 4.1
Examples of two-classes classification problems in a 2-dimensional space.
a
Linear bound-
ary.
represent the informative direction and the redundant direction respectively,
b
Closed
boundary,
c
Piecewise linear boundary
ʱ
and
ʲ
Search WWH ::
Custom Search