Civil Engineering Reference
In-Depth Information
Table 7.1
Metrics for the assessment of classifi cation results
Performance measure
Formula
()
−
()
PA
PE
PE
Kappa statistic
()
1
−
pa p a
−+ − +
p a
−
Mean absolute error
1
1
2
2
n
n
n
(
)
2
(
)
2
(
)
2
Root mean squared error
pa
−
+−
p a
++ −
p a
1
1
2
2
n
n
n
pa
−++ −
−++ −
p a
Relative absolute error
1
1
n
n
aa
a a
1
n
1
∑
where
a
=
a
i
n
i
(
)
2
(
)
2
pa
−
++ −
p a
Root relative squared error
1
1
n
n
(
)
++ −
2
(
)
2
aa
−
a a
1
n
Confusion matrix
classified as
a
b
classified as
a
b
a = Yes
x
1
x
2
a = Yes
p
11
p
12
p
1
·.
x
3
x
4
p
p
22
p
2
·.
b = No
b = No
21
x =
x
1
+
x
2
+
x
3
+
x
4
p
·
1
p
·
2
1
p
11
=
x
1
/x,
p
12
=
x
2
/x,
p
21
=
x
3
/x,
p
22
=
x
4
/x
p
·
1
=
p
11
+
p
21
,
p
·
2
=
p
12
+
p
22
P
(A) =
p
11
+
p
21
p
1
· =
p
11
+
p
12
,
p
·
2
=
p
21
+
p
22
P
(E) =
p
·
1
p
1
· +
p
·
2
p
2
·
7.2
Calculation of kappa statistic.
mean-squared error is simply the square root of the mean-squared error.
Relative absolute error is the total absolute error made relative to what the
error would have been if the prediction simply had been the average of the
actual values. Relative squared error is the total squared error made rela-
tive to what the error would have been if the prediction had been the
average of the absolute value. The square root of the relative squared error
is taken to give the same dimensions as the predicted values themselves
(Witten
et al.
2011). For these error measurements, the smaller the value,
the better the performance of classifi cation.
In the two-class case, a single prediction has four possible outcomes as
shown in Table 7.2. The true positive (TP) and true negative (TN) are
correct classifi cation (Witten
et al.
2011). A false positive (FP) occurs when
the output is incorrectly predicted as negative while a false negative (FN)
Search WWH ::
Custom Search