Information Technology Reference
In-Depth Information
Of the cases used to create the model, 9412 of the 9508 normal attacks are
classi
ed correctly (99 %) and 8142 of the 8215 anomaly attack types are classi
ed
correctly (99.1 %). Overall, 99.0 % of the training cases are classi
ed correctly,
corresponding to the 1 % incorrect shown in the Table
4
of model summary. Thus
the model generates a better classi
cation by correctly identifying a higher per-
centage of the cases. Classi
cations based upon the cases used to create the model
tend to be too
ated. The
holdout sample facilitates to validate the model; here 98.8 % of these cases were
correctly classi
“
optimistic
”
in the sense that their classi
cation rate is in
fl
ed by the model. This suggests that, overall, the proposed model is
in fact correct.
In Table
6
the model summary shows a couple of positive signs:
The percentage of incorrect predictions is roughly equal across training, testing,
and holdout samples. The estimation algorithm stopped because the error did not
decrease after a step in the algorithm. This further suggests that the original model
did not over trained.
The confusion matrix in Table
7
shows that, the network does excellent at
detecting anomaly than normal attacks. The detection rate and overall accuracy of
Table 6 Confusion matrix
Sample
Observed
Predicted
a
n
Percent correct (%)
Training
a
7,019
59
99.2
n
70
7,981
99.1
Overall percent
46.9 %
53.1 %
99.1
Testing
a
3,431
31
99.1
n
53
4,044
98.7
Overall percent
46.1 %
53.9 %
98.9
Holdout
a
1,190
12
99.0
n
17
1,284
98.7
Overall percent
48.2 %
51.8 %
98.8
Dependent variable: class
Table 7 Model summary
Training
Cross entropy error
389.173
% Incorrect predictions
0.9 %
1 consecutive step(s) with no decrease in error
a
Stopping rule used
Training time
00:00:25.563
Testing
Cross entropy error
246.806
Percent incorrect predictions
1.1 %
Holdout
Percent incorrect predictions
1.2 %
Dependent variable: class
a
Error computations are based on the testing sample
Search WWH ::
Custom Search