Database Reference
In-Depth Information
3.4.5 Beyond WLCG: Data Management
Use Cases in EGEE
The data management use cases of many other disciplines for which the EGEE
infrastructure is intended have a lot in common with those of the LHC ex-
periments, which have been the main driving force behind the development
of the various data management services and client utilities available today.
Other disciplines also pose additional requirements that matter less to the
LHC experiments, at least for the time being:
Fully encrypted communication and storage for much-improved privacy. For
example, in biomedical research it is vital that no data be accidentally
exposed, whereas LHC expermenters do not mind if the vast majority of
their data happens to be world-readable. The expermenters do not want
their high-level analysis results to be exposed, but such results typically
can be stored and processed locally without the need for grid services.
On the other hand, encrypting and decrypting their huge volumes of low-
level data on the fly probably would still be an unacceptable overhead
for a long time. For the biomedical community, encrypted storage has
been developed as a plug-in for the DPM storage element and the GFAL
client suite. An additional necessary component is the Hydra distributed
key service.
30
Standard POSIX access to data served by storage elements. Most of the
EGEE communities have legacy applications that expect to read and
write files through standard POSIX I/O instead of GFAL,
17
ROOT.
26
XROOTD,
20
or server-dependent protocols like RFIO
4
and DCAP.
31
It
was fairly straightforward for a classic SE to make its data available to
a local batch farm via NFS, but this has not been possible for SRM
services. An SRM should be able to replicate hot files (i.e., files that
are accessed frequently) and direct clients to any disk server in a load-
balanced way. It should also be able to clean up a replica without the risk
that a client might still be reading it. These problems can be overcome,
though. For example, the main objective for the StoRM implementation
of SRM v2.2 is to allow secure POSIX access by legacy applications,
typically through a cluster file system like GPFS and with just-in-time
access control lists, so that a file is only accessible for a duration that
a client has to negotiate with the SRM in advance. All of the SRM
implementations in use on the EGEE infrastructure have expressed in-
terest in developing NFSv4 interfaces, which would allow for standard
POSIX access by clients without reducing vital server functionality. Fi-
nally, a transparent remote I/O library like Parrot (see Chapter 4) may
be enhanced with plug-ins for some of the popular protocols that can
be served. This will not help statically linked applications, though.
Availability of client utilities on many platforms. To optimize the use of
available resources, the HEP experiments and laboratories have moved
