Database Reference
In-Depth Information
Custom Web
Applications
Dashboard
Application
End-User
Interfaces
Provenance
& Workflow
Management
Provenance
Recording API
Provenance
Query API
Workflow
Storge API
Provenance
Store
Kepler Scientific
Workflow System
Workflow
Automation API
Workflow
Automation
Simulation
(Supercomputer/Cluster)
Analytics
(“post-processing”)
Workflow
Deployment
Figure 13.5
Overview of the SDM provenance framework.
created them. Other uses are increased workflow robustness (cf.
Section 13.3.2) and improved eciency upon rerunning only the affected
parts after modifying some workflow parameters or inputs.
Workflow evolution
, which captures changes over time to the workflow
description (including parameter changes). It is particularly important
to track workflow evolution as part of exploratory workflow design, when
there are many cycles of workflow modifications and workflow runs.
System environment
information, which captures data about the system
that executes the workflow, and its environment, for example, the ma-
chines, operating systems, compiler versions, job queues, and so forth
that were used. It is important to capture such information since, in
practice, results will depend on the system environment that a workflow
executes in.
Kepler has been extended to record and store various forms of prove-
nance information.
64
,
73
,
74
Depending on the settings of the Kepler prove-
nance recorder, data may be recorded for all actors in the workflow, or some
subset, for example, only top-level composites. The recording API also sup-
ports recording of information from components external to Kepler (e.g., from
Python or shell scripts that are invoked by an actor).
A provenance query API provides a (read-only) mechanism to retrieve
provenance information from the provenance store, for example, a call-back
mechanism to notify applications (such as a Web-based workflow monitor-
ing dashboard) during workflow execution. In addition to providing current
workflow status, authorized users and applications can query the provenance
store about past executions via an SQL interface, thus supporting provenance
analytics.
