Information Technology Reference
In-Depth Information
averages across all three datasets were 0.6184 for Super Wide MLR compared to 0.6096 (significance
0.1066) for ES and compared to 0.5651 (significance 0.0000) for Super Wide SVM.
From these results, we found that the MLR performed slightly better than ES for the two manufactur-
er's datasets, but performed worse than ES when considering all three datasets. Although we could not
clearly state whether a linear version of the Super Wide Model was better than the traditional and simple
ES, we did find that results using Super Wide Support Vector Machine Model were significantly better
than the multiple linear regression, its linear counterpart, which would indicate that the performance
increase identified was not isolated to only the Super Wide model. Additionally, since in almost all of
the cases performance of both non-linear and linear alternative techniques (without Super Wide data)
were much worse than the exponential smoothing, the high quality modeling performed by the Super
Wide Support Vector Machine seemed to be a result of the combination of the Super Wide data and the
Support Vector Machine and not a result of either method applied separately.
Average Performance
Hypothesis 1 stated that Machine Learning - based techniques used at the end of the supply chain to
forecast distorted customer demand as experienced by a manufacturer would have better average per-
formance than traditional techniques. To test this hypothesis, we examined the difference between the
average error of traditional forecasting techniques and ML - based techniques. By taking the average
error of the control and treatment group we can evaluate if ML in general presents a better solution.
As noted earlier, the Trend technique provided extremely poor forecasting and the error measure-
ments produced were extreme outliers. However, they were not outliers in the sense of being measure-
ment errors, they were forecasted values as defined by the algorithm and were retained in the average
calculation. Additionally, the Trend forecasting techniques had to be retained in the experiment because
it is representative of what practitioners use. According to Jain (2004c), averages and simple trend are
used 65% of the time.
Using the results of the experiments on chocolate (Table 1) and toner cartridge manufacturer (Table
2), the average error in the control group was 1.5014 and the average error in the treatment group was
0.8356. It was not feasible to take all of the error points of the test sets for each forecasting technique
and compare those of the control group with the treatment group since there would be 52800 observa-
tions for the control group and 44000 observations for the treatment group. However, considering the
large difference between the averages and the large number of observations, the t-test would have had
a very high significance value and we considered this hypothesis as supported.
Accordingly, without any specific information on which traditional and ML techniques are best or
if there is a lack of experience and knowledge related to traditional and ML techniques, one would be
more likely to be better off in choosing a random ML solution to induce the lower expected error.
r ank Performance
In hypothesis 2, we stated that ML - based techniques used at the end of the supply chain to forecast
distorted customer demand as experienced by a manufacturer would have better ranked-performance
than traditional techniques. By taking the average rank of the control and treatment group we evaluated
if ML in general presents a better solution. Using the results of the experiments on chocolate (Table
1) and toner cartridge manufacturer (Table 2), the average rank in the control group was 11.25 and the
Search WWH ::
Custom Search