Database Reference
In-Depth Information
the Change Detection Procedure will reveal the changes in these periods
only and not in any other period.
3.2.
Results — Part 1 (Hit Rate and False Alarm Rate)
Based on the 24 change detection trials the following accumulated results
have been obtained, as described thoroughly in Tables 5 and 6:
•
All artificial changes in the “patterns” (rules) relating the input variables
to the target variable were detected by the CD (5%) procedure. The
average detection rate of CD was 100% meaning that all changes were
recognized as significant by the procedure.
•
Two trials generated by an artificial change in the target variable were
detected by the CD procedure. The average detection rate of CD in these
trials was 100%.
•
According to the XP hypothesis testing, 50% of changes in the candidate
input variables produced a change in the target variable, resulting in an
average detection rate of 100% for input attributes and 98% for the target
attribute.
•
All the changes, which were mainly introduced to affect the target vari-
able, were recognized as significant by the XP procedure.
•
All the changes, which were mainly introduced to affect the relationship
between the target and the candidate input variables, were recognized as
significant by the XP procedure, with an average of 100% detection rate.
•
The actual 2nd type error (false negative) rate of the change detection
methodology in this experiment is 0%. No change was left undetected by
the methodology.
•
The actual 1st type error (false alarm) rate of the change detection
methodology in these experiments is up to 6%. When implementing the
methodology in trials which do not include changes, the CD estimator
did not produce false alarms (0% error rate) and the XP estimator failed
to produce accurate estimation in only five out of 84 tests (5.9%).
To conclude this part of the experiments with various possible changes
in the data mining model, the following assumptions are validated by the
results which were described above: a major change can cause secondary
effects and generate other changes. The Change Detection procedure can
detect these changes with theoretically and actually low Type 1 and Type 2
error rates.
