Database Reference
In-Depth Information
3.5.2 The Earth System Grid
The purpose of the ESG is to provide the worldwide community of climate sci-
entists access to climate modelling data from multiple sources. This includes
data from the Community Climate System Model (CCSM), 34 the Intergov-
ernmental Panel on Climate Change (IPCC) 35 model, the Parallel Ocean Pro-
gram (POP), 36 and others. In the current version of ESG, the data is stored
in multiple distributed sites including NCAR and several Department of En-
ergy Laboratories: LANL, LBNL, LLNL, and ORNL. As of the end of 2008,
the ESG has about 9,000 registered users, over 200 TB of data that can be
accessed by users, and so far supported over 66 TBs of data downloaded.
The ESG is built using a collection of tools for finding the desired data using
metadata catalogs and search engines, for requesting aggregation of the data,
and for moving subsets of the data to users. The current distributed organi-
zation is shown in Figure 3.7 where only the SRMs, the storage components
(disks, MSS) and the GridFTP servers are shown. The SRMs call the GridFTP
servers to perform the file transfers. The SRM at the portal has a large disk
cache shared by all active clients. It allocates space for each user and brings
in files from any of the other sites, either files residing on disks or files that
SRMs bring into disks from MSSs if necessary. So far, file streaming has not
been used in this version of ESG since the number of files allowed to be down-
loaded was limited. However, as the number of files requested grew, the disk
cache became clogged and had to be enlarged. In anticipation of much larger
use the next version of ESG, which will be a federated system with multiple
portals and data nodes all over the world, will use file streaming in order to
have more effective use of the shared disk caches and avoid clogging the disks.
Clients
NCAR
LBNL
HPSS
High Performance
Storage System
ESG Portal
SRM
Storage Resource
Management
Disk
ORNL
SRM
Storage Resource
Management
SRM
Storage Resource
Management
gridFTP
Server
gridFTP
Server
Disk
HPSS
High Performance
Storage System
LANL
Disk
SRM
Storage Resource
Management
Disk
gridFTP
Server
SRM
Storage Resource
Management
gridFTP
Server
MSS
Mass Storage System
Disk
Figure 3.7
The Earth System Grid use of SRMs for file streaming to clients.
Search WWH ::




Custom Search