Database Reference
In-Depth Information
heuristic fails when a MapReduce configuration does not match the cluster topol-
ogy. Furthermore, MARLA could improve by adapting to a cluster's topology
without sacrificing the late binding of tasks to workers; we plan to study this as
future work.
Fig. 10.
This contour plot shows the effects of varying two kinds of nodes within a
cluster with respect to computation time. In this case, the effect of 24 tasks in a
24 node cluster that assumes 32 sub-tasks for each task. The X-axis shows the percent-
age of the cluster that has been upgraded to Faster nodes, while the Y-axis shows the
percentage of the cluster that has been upgraded to Fastest nodes. Impossible points
have been interpolated. The solid lines indicate the trends in the data.
Figure
11
shows results of dividing work onto 72 tasks (three per worker), and
shows that upgrading impacts application turn-around time for smaller task gran-
ularities and for systems whose MARLA number-of-cores parameter is set prop-
erly. With 72 tasks and 32 sub-tasks per task, and more than 12.5 % Fastest nodes,
execution time drops below 100 s. Further, configurations that split into 32 nodes
perform better than when tasks split into only 8 subtasks, because the Fastest
nodes can use all 32 cores. In this case, performance improves more effectively
when the most powerful nodes in the cluster are using effectively utilized.
This section's results indicate:
- The one task per worker heuristic combats
performance-heterogeneity
for the
configurations we tested, with three discrete levels of worker performance.
- A mis-configuration of a MapReduce framework that is not fully aware of clus-
ter topology can reduce the number of configurations that provide improved
performance as the cluster is upgraded.
- The addition of more tasks to the pool of tasks that needs to be completed
allows for a MapReduce framework to be configured so that it takes full
advantage of the Fastest nodes in the cluster and still sees improved turn-
around time for most cluster configurations.
Search WWH ::
Custom Search