Increasing the Accuracy of Software Fault Prediction Using Majority Ranking Fuzzy Clustering* - Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing - page 181

6 Experimental Results and Analysis

In order to evaluate our work, we used two well-performed learning methods based
on the literature reviews namely random forest and naïve bayes [2, 13, 29]. This
section composed of two sets of experiments. Tables 1 to 7 present the prediction
error analysis for the proposed fuzzy approach as compared to other approaches,
namely, naïvebayes (NB), random forest (RF), Alan and Catal proposed approach
with naïve bayes(ACN) [23], Alan and Catal proposed approach with random For-
est (ACF) [23], our proposed fuzzy majority ranking approach with outliers
(FMRT), and our proposed fuzzy majority ranking approach with outliers removal
(FMR).
In order to answer research question 1, we look at FMRT rows in Tables 1 to 7.
As it can be seen, our proposed model performed almost same as naïve bayes and
random forest with slight difference in error, FNR, and FPR values in NASA sets.
FNR value in CM1 is improved with 10 and 20 percent compared to naïve bayes
and random forest respectively. In Turkish set, AR4, on the other hand, FNR and
error rate were increased considerably while FPR value has decreased. According
to the results, we can conclude that our proposed method performed as well as two
well performed learning algorithm but it does not outperformed them generally, so
we accept null hypothesis 1.

Table 1 Results on KC1

Method FPR FNR Error

NB 0.09 0.63 0.17

RF 0.06 0.73 0.16

ACN 0.07 0.16 0.06

ACF 0.01 0.31 0.03

FMRT 0.06 0.74 0.16

FMR 0.03 0.13 0.03

Table 2 Results on KC2

Method FPR FNR Error

NB 0.05 0.53 0.15

RF 0.09 0.53 0.18

ACN 0.06 0.10 0.07

ACF 0.03 0.27 0.06

FMRT 0.06 0.52 0.17

FMR 0.04 0.18 0.06