Information Technology Reference
In-Depth Information
The conditional probability in formula (6.19) can be gained using maximum
likelihood estimation:
count
(
v
∧
c
)
j
i
P
(
v
|
C
)
=
(6.20)
j
i
count
(
c
)
i
To avoid zero probability, if the actual conditional probability is zero, it is
assigned to be 0.5/
, where N is the total number of examples.
Suppose that there are only two classes, namely class0 and class1, and
N
a
1
, …,
b
=
P C
(
=
0)
b
=
P C
(
=
1)
=
1
−
b
a
k
represent features of test set. Let
,
,
0
1
0
p
=
P A
(
=
a
|
C
=
0)
p
=
P A
(
=
a
|
C
=
0)
,
, then:
j
0
j
j
j
1
j
j
k
∏
=
p
=
P
(
C
=
1
|
A
=
a
∧
?
∧
A
=
a
)
=
(
p
)
b
/
z
(6.21)
1
1
k
k
j
1
1
j
1
k
∏
=
q
=
P
(
C
=
0
|
A
=
a
∧
?
∧
A
=
a
)
=
(
p
)
b
/
z
(6.22)
1
1
k
k
j
0
0
j
1
where
is a constant. After taking logarithm on both sides of the above two
equations, we subtract the second equation from the first one and get:
z
k
Ã
=
log
p
−
log
q
=
(
log
p
−
log
p
)
+
log
b
−
log
b
(6.23)
j
1
j
0
1
0
j
1
w
=
log
p
−
log
p
,
b
=
log
b
−
log
b
Here, let
,
the above equation is
j
j
1
j
0
1
0
written as:
k
Ã
=1
log
(
−
p
)
/
p
=
−
w
j
−
b
(6.24)
j
After taking exponential on both sides of equation (6.24) and rearranging, we
have:
1
k
p
=
(6.25)
−
Ã
w
−
b
j
1
+
e
j
=
1
To calculate this value, we assume that feature
A
j
has
v
(
j
) possible values. Let
w
=
log
P A
(
=
a
|
c
=
1)
j
'
'
jj
jj
'
log
P A
(
=
a
|
c
=
0)
(1
≤
j
≤
v j
( ))
(6.26)
j
'
jj
We have:
1
P
(
C
(
x
)
=
1
=
(6.27)
k
j
v
j
(
'
j
)
Ã
Ã
−
(
I
(
A
(
x
)
=
a
)
w
−
b
j
'
'
1
+
e
=
1
jj
jj
