Information Technology Reference
In-Depth Information
Table 2.
Summary of results averaged by strategy
Strategy Speedup Cost/task Usage [%] Failures
ScaF
33.0
0.73
90.4
N/A
SchF
10.7
0.75
89.5
N/A
SIAA
47.9
0.48
77.7
100.9
SIAA outperforms the remaining strategies in terms of speedup and cost per
task. These results corroborate that SIAA permits faster and cheaper executions
than its competitors. In the downside, it can be also seen that SIAA evidences
a lower percentage of instances usage (up to 22.3% of time wasted) and number
of 100.9 failures (about 10.1% of tasks). However, the inferior use of instances
and the (relatively) large number of failures do not prevent the strategy from
achieving high time and cost savings when compared with ScaF and SchF. The
following section provides a deeper insight in the aspect of task failures by ana-
lyzing the robustness of SIAA in several uncertainty scenarios.
4.2 Robustness Analysis
The previous experiment evidenced the advantages of using SIAA by compar-
ing its performance with other autoscaling strategies. This section analyzes the
robustness of SIAA in terms of the number of failures and speedup for a wide
number of settings involving different balances of on-demand/spot instances and
different errors affecting the bid price prediction methods.
Although for this second experiment the same type of workflows were used
we focused on applications around 100 tasks to limit the size of the experiment.
All the simulations were carried out using a 20% error for tasks running time
and instances lag time. In all cases the budget was set to $30. Figure 4 presents
the number of task failures and speedups according to scenarios defined by:
1. The
spotsRatio
parameter (percentage of the total budget assigned to spot
instances) varying from 0
,
20
,...,
100% (horizontal axis), and
2. The
bidError
parameter (error affecting the bid price prediction methods)
(vertical axis). Errors vary from 0 (an hypothetical perfect predictor) and a
48.5% predictor error.
Task Failures.
From the top figure it can be seen that as the
spotsRatio
parameter increases, the number of failures augments. This is because having a
smaller proportion of on-demand instances makes the critical tasks more prone
to run on spot instances and therefore are more likely to fail. By looking on the
other axis, as the prediction error increases the number of failures also increases.
Larger errors on the prediction of the optimal bid price increment the probability
of failure contributing to an increase of the total number of failures.
