Information Technology Reference
In-Depth Information
standard to have a way to specify CBP-speci
c settings, for example the name of
the selected CBP application or the resources to be used for the job
s execution.
The DCI Bridge performs submission of jobs according to the following steps:
'
1. DCI Bridge collects all the input
file or workflow storage
(WFS) services of gUSE based on the information stored in the incoming JSDL.
2. A new CBP job is created using the Java API of the CBP.
3. Different properties of the CBP job are set based on the incoming JSDL, like the
job
files either from the
s name, the application to be run, or the resource to be used for execution.
DCI Bridge asks the CBP to collect the output
'
files of the job after its termi-
nation as an archive (resulting in much faster job
finishing times compared to
files are transferred back one-by-one).
4. The DCI Bridge packs the input
the case when output
files collected in step 1 into an archive. Once
the archive is ready, the DCI Bridge invokes the CBP API to add the archive to
the CBP job (created in step 2) as an input
file. This also makes the CBP API
upload the archive to the cloud storage that is attached to the job
'
is cloud
resource, and register the location of the
file to the CBP job (created in step 2).
'
This means that the input archive
is data is not transferred from the DCI Bridge
to cloud storage through the CBP service, but the API call uploads the job
'
s
input archive directly to the cloud storage from the DCI Bridge service.
5. Once all the properties are set and the input archive is uploaded, the job is
submitted to the CBP service.
According to our experiences, handling jobs with the above input and output
archiving methods is very ef
files. For
example, the whole lifecycle of a short job (running for a few seconds) that pro-
duced 100 output
cient for jobs with many input or output
files lasts for almost 15 min without, and for 3 min with output
archive usage. The metrics are the same for short jobs that have similar number of
input
files. This behavior follows from the way that the CBP manages the data
related to the jobs:
first, the input
files have to be added to the job, which results in
uploads to the selected resource
s storage through a redirection from the CBP
service. Once the virtual machine is started, the
'
files are transferred from the storage
to the virtual machine. Output
file handling happens similarly to input
file handling,
but in the opposite direction.
Status of submitted jobs is periodically updated using the CBP Java API. The
returned CloudBroker-speci
c job status is matched against the relevant OGSA-
BES job status. Once a job reaches a terminal status, its output
files are handled as
follows by the DCI Bridge:
1. The output archive
file is fetched into a temporary directory on the DCI Bridge
service
'
s machine using the CBP Java API. Just like in case of the input
le
le
resides, so the portal will fetch the results directly from the cloud Storage, and
not through the CBP service used.
2. The content of the output archive is decompressed into the job ' is working
directory on the DCI Bridge service.
archive, this API call redirects the caller to the cloud Storage where the
Search WWH ::




Custom Search