Database Reference
In-Depth Information
Shortest Job First (SJF) Scheduling:
The data placement job which is
expected to take the least amount of time to complete will be executed
first. All data placement jobs except the transfer jobs have job com-
pletion time in the order of seconds, or minutes in the worst case. On
the other hand, the execution time for the transfer jobs can vary from
a couple of seconds to a couple of hours or even days. Accepting this
policy would mean nontransfer jobs would be executed always before
transfer jobs. This may cause big delays in executing the actual transfer
jobs, which defeats the whole purpose of scheduling data placement.
Multilevel Queue Priority Scheduling:
In this case, each type of data
placement job is sent to separate queues. A priority is assigned to each
job queue, and the jobs in the highest priority queue are executed first.
To prevent starvation of the low-priority jobs, the traditional aging tech-
nique is applied. The hardest problem here is determining the priorities
of each data placement job type.
Random Scheduling:
A data placement job in the queue is randomly
picked and executed without considering any criteria.
4.3.2.1
Auxiliary Scheduling of Data Transfer Jobs
The above techniques are applied to all data placement jobs regardless of
the type. After this ordering, some job types require additional scheduling
for further optimization. One such type is the data transfer jobs. The transfer
jobs are the most resource consuming ones. They consume much more storage
space, network bandwidth, and CPU cycles than any other data placement
job. If not planned well, they can fill up all storage space, thrash and even
crash servers, or congest all of the network links between the source and the
destination.
Storage Space Management.
One of the important resources that needs
to be taken into consideration when making scheduling decisions is the avail-
able storage space at the destination. The ideal case would be the destination
storage system supports space allocations, as in the case of NeST,
1
and be-
fore submitting a data transfer job, a space allocation job is submitted in the
workflow. This way, it is assured that the destination storage system will have
sucient available space for this transfer.
Unfortunately, not all storage systems support space allocations. For such
systems, the data placement scheduler needs to make the best effort in order
to not overcommit the storage space. This is performed by keeping track of
the size of the data transferred to and removed from each storage system that
does not support space allocation. When ordering the transfer requests to that
particular storage system, the remaining amount of available space, to the best
of the scheduler's knowledge, is taken into consideration. This method does
not assure availability of storage space during the transfer of a file, since there
can be external effects, such as users who access the same storage system via
