Database Reference
In-Depth Information
3.6 Conclusions and Future Work
Storage management is one of the most important enabling technologies for
large-scale scientific investigations. Storage resource managers (SRMs) pro-
vide the technology needed to manage the rapidly growing distributed data
volumes, as a result of faster and larger computational facilities. SRMs are
software components whose function is to provide dynamic space allocation
and file management on shared storage systems. They call on transport ser-
vices to bring files into their spaces transparently and provide effective sharing
of files. SRMs are based on a common specification that emerged over time
and evolved into an international collaboration. This approach of an open
specification that can be used by various institutions to adapt to their own
storage systems has proven to be a remarkable success. SRMs are being used
in production by multiple facilities to provide uniform interfaces to various
storage systems at many sites in the United States, Canada, Europe, Asia,
and elsewhere. They are being used in several application domains, quite ex-
tensively in high-energy and nuclear physics experiments, the Earth System
Grid, and other application domains, including fusion modeling, biology, and
medical applications.
Based on past experience we have identified several features that require
further development and coordination. These include sophisticated aspects
of resource monitoring that can be used for performance estimation, autho-
rization enforcement in coordination with virtual organizations (VOs), and
accounting tracking and reporting for the purpose of enforcing individual or
group quotas in the use of SRM resources. Performance estimation is very
important for the clients to decide whether to launch a request to a particular
SRM, especially when that SRM is highly loaded. Such a task is nontrivial,
in that an SRM has to estimate not only its own load (based on queue mon-
itoring), but also take into account external performance estimations, such
as the time to get files in or out of MSSs, or from another site over the net-
work. Another item to be considered for support by SRMs is advance of space
reservation. Currently, SRMs only respond to a space reservation request at
the time the request is made. It is much harder to manage reservations for
periods in the future. However, such provisioning capabilities have started to
be supported by networks dedicated to research communities such as Energy
Sciences Network (ESnet) and Internet2. Advance network provisioning and
compute resource reservations will have to be coordinated with advance space
reservation when supported by SRMs, in order to plan and perform complex
coordinated tasks successfully. Finally, the issue of enforcing authorization is
quite complex, and requires a foolproof, secure way of coordinating with a
VO. In general, given that the SRM back-end storage resources are shared by
multiple users, the quota assigned to a user (measured in gigabyte-minutes,
for example) has to be centrally coordinated with a VO, or predetermined, in
