Information Technology Reference
In-Depth Information
As it shows, the submit host consists of Workflow Mapper which maps abstract
workflows to concrete workflows that are dependent on execution sites, Clustering
Engine which merges small tasks into a large job such that the scheduling overhead is
reduced, and Workflow Engine which handles the data dependencies and local sche-
duler and that only releases free tasks to Clustering Engine. The Execution Site con-
sists of Remote Scheduler which is used to match jobs to a worker node based on the
criteria selected by users, Worker Nodes, Failure Generator which is introduced to
inject task/job failures at each execution site during the simulation and Failure Moni-
tor which collects failure records to return these records to Clustering Engine to adjust
the scheduling strategies dynamically [19].
In actual operation process, as the workflow is so numerous, even contains tens of
thousands of tasks, while we usually have only dozens of computing nodes, task clus-
tering needed to use for polymerizing similar tasks at this moment, and form clustered
job accordingly, which generally called job. Each job include several tasks, and inte-
grally submit to operating environment, in this way can save a lot of submission de-
lay, and open and execute the clustered job separately when a certain computing node
is available.
WorkflowSim is used for validating Graph algorithm, distributed computing,
workflow scheduling, resource provisioning and so on. In addition, WorkflowSim is
an open source workflow simulator that has been hosted on GitHub
3
. Compared to
CloudSim and other workflow simulators, WorkflowSim provides support of task
clustering that merges tasks into a cluster job and dynamic scheduling algorithm that
jobs matched to a worker node whenever a worker node become idle. A series of
popular workflow scheduling algorithm (e.g., HEFT, Min-Min, and Max-Min) and
task clustering algorithms have been implemented in WorkflowSim. Users can speci-
fy different criteria to optimize the overall performance.
4.3
SwinDeW-C and SwinFlow-Cloud
Instance-intensive application is one ubiquitous workflow application in real life, but
traditional workflow systems cannot give an enough support to these applications.
Therefore, the group led by Yun Yang professor has proposed the concepts of
instance-intensive workflow, and focused on research of the design of system archi-
tecture, scheduling algorithms. The cloud workflow systems have gained rapid devel-
opment, the typical of which are SwinDeW-C based on SwinDeW-G from Swinburne
University of Technology, Cloud based on Hadoop from University of Waterloo,
Cloudbus Engine based on Gridbus from University of Melbourne. The design of
cloud workflow system architecture is following:
The workflow system architecture has improved from centralized architecture to de-
centralized architecture. But centralized architecture is still the most popular paradigm
in today's workflow community because it is simple and easy to rapidly implement a
prototype or product to support workflows. The client-server model also is a typical
3
https://github.com/WorkflowSim/WorkflowSim-1.0
