Database Reference
In-Depth Information
According to the change detection methodology, during all six consecu-
tive months there was no significant change in the rules describing the rela-
tionships between the candidate and the target variables (which is our main
interest). Nevertheless, it is easy to notice that major changes have been
revealed by the XP statistic in distributions of most target and candidate
input variables. One can expect that variables with a large number of values
need greater data sets in order to reduce the variation of their distribution
across periods. However, this phenomenon has not affected the CD statistic.
An interesting phenomenon is the increasing rate of the CD confidence
level from month 2 to month 6. In order to further investigate whether a
change in frequency distribution has still occurred during the six consecu-
tive months without resulting in a significant CD confidence level, we have
validated the sixth month on the fifth and the first months. Table 11 and
Figure 3 describe the outcomes of the change detection methodology.
Implementing the change detection methodology by validating the sixth
month on the fifth and the first month did not produce contradicting results.
That is, the CD confidence level of both months ranges only within
8%
from the original CD estimation based on the all five previous months.
Furthermore, although XP produced extremely high confidence levels indi-
cating a drastic change in the distribution of all candidate and target vari-
ables, the data mining model was not affected, and it kept producing similar
validation error rates (which were statistically evaluated by CD).
The following statements summarize the case study's detailed results:
±
•
Our expectation that in the 'Manufacturing' database, there are no sig-
nificant changes in the relationship between the candidate input variables
and the target variable over time, is validated by the change detection
Table 11. Outcomes of XP by validating the sixth month on the fifth and the first month
in 'Manufacturing' database (
p
-value).
CAT MRKT Duration
Time
Quantity Customer Target
GRP
Code
to Operate
GRP
Metric XP
domain
18
19
19
19
15
18
2
(1
−
p
-value) month
5 validated by
month 6
100% 100%
100%
100%
100%
100%
98.4%
month
1 validated by
month 6
100% 100%
100%
100%
100%
100%
100%
months 1 to
5 validated by
month 6
100% 100%
100%
100%
100%
100%
63.1%
