Closing the Gap between Cloud Providers and Scientific Users - Cloud Computing with e-Science Applications - page 123

defined execution, and they are only valid in the context of that execution.
A description of the proposal to handle each data type is included in the
following sections.
e-Clouds users can store, manage, and share their personal files within
the platform. This includes the ability to organize them in a file system, with
a directory structure, and generating publicly accessible URLs (uniform
resource locators) for each one. To address these needs, the files reside in the
storage service provided by the public IaaS, and the corresponding meta-
data are stored in a relational database. Every execution input and output
file will be stored as part of the e-Clouds file system.
6.4.2.2 Resource Manager
Resource manager is the piece of software in charge of job scheduling
and cluster control. It has the responsibility of starting and stopping new
machines on demand, taking into account the pending jobs. It assigns a
certain amount of workload to each machine in an effort to minimize the
total costs and time. In addition to the management functions, the RM serves
as a central communication channel between the front end and any machine
in a cluster.
6.4.2.3 Queues for Asynchronous and Scalable Communications
Due to the dynamic and flexible scaling of the cloud infrastructure, commu-
nication between cluster nodes and other e-Clouds components is achieved
using queues. These queues will be accessible from all components in the
architecture and will offer a reliable and scalable asynchronous messaging
system. An important benefit obtained with the use of queues is the possibil-
ity to buffer user jobs.
6.4.2.4 Agent in Processing Machines
The agent is a control program that resides in every machine that runs as
part of an e-Clouds execution. It is in charge of managing the local job execu-
tions and communications of a machine with the RM. As part of its responsi-
bilities, it handles app installation and configuration, launches assigned jobs,
and monitors the overall execution progress, communicating any updates.
6.4.2.5 Billing
A billing system is included to provide information about the cost of the
resources consumed. An event-based approach is taken to calculate resource
usage for the current period. This means that the system is capable of track-
ing each event that somehow has an impact on the total costs and records the
pertinent information. Metrics such as machine hours, data transfer to and