Information Technology Reference
In-Depth Information
average rank in the treatment group was 11.8. Therefore, on average, ML techniques ranked worse
than the traditional ones. Thus, the hypothesis is rejected and a rigorous testing is not required as the
difference is in the direction opposite to the expected one. If the Statistics Canada data are included,
the averages rank of both the control and the treatment group were 11.5, which also implies rejection
of the hypothesis.
Accordingly, without any specific information on which traditional and which ML techniques are
best or if there is a lack of experience and knowledge related to traditional and ML techniques, one can-
not just select any ML method and generally expect it to perform better (i.e. in terms of the “winning”
method, not based on the magnitude of error) than a random traditional technique.
Comparison of the Best ML and Best Traditional Technique
Hypothesis 3 stated that the best ML - based technique used at the end of the supply chain to forecast
distorted customer demand as experienced by a manufacturer will have better performance than the best
traditional technique. To statistically compare the performance of the two we performed a t-test. The
difference between the support vector machine error average of 0.7154 and the automatic exponential
smoothing error average of 0.7516 had a p-value of 0.0000. Across the three datasets the support vector
machine error average was 0.5651 and the automatic exponential smoothing error average was 0.6096,
a statistically significant difference with p-value of 0.0000. Thus, we can conclude that the best ML
approach has performed significantly better than the best traditional approach.
Sensitivity Analysis
Using the Super Wide data for the Support Vector Machine modeling permitted an analysis of much
more data simultaneously and thus produced a very high historical window size while still having a
very large dataset to learn from. As a result of this, we set the historical window size to 50% of the
history. However, we investigated whether 50% was the correct setting and whether this setting had an
impact on the performance of the model. Although a historical window size of 50% seemed normal,
Table 4. Sensitivity analysis of window size
Data set
Window
MAE
Chocolate
0.40
0.7806
Chocolate
0.50
0.7717
Chocolate
0.60
0.7703
Toner Cartridge
0.40
0.6770
Toner Cartridge
0.50
0.6777
Toner Cartridge
0.60
0.6846
Statistics Canada
0.40
0 3903
Statistics Canada
0.50
0.4547
Statistics Canada
0.60
0.4414
Search WWH ::
Custom Search