Hardware Reference
In-Depth Information
Adaptable I/O System (ADIOS) to bridge this gap between complex dis-
tributed in-memory data structures to the underlying parallel file system.
These libraries provide both a higher-level semantic than a simple byte stream
while also optimizing parallel I/O operations to the underlying parallel file
system environment.
Complementing these high performance parallel I/O environments are
large-scale archival storage systems such the High Performance Storage Sys-
tem (HPSS). These systems provide HPC facilities with the ability to manage
hundreds of petabytes of tape and/or disk storage for long-term data storage.
These systems balance performance and capacity by supporting automated
tiering of data between tape and disk while providing high performance read-
ing and writing to and from the archive via parallel data movement nodes. In
some cases, these large-scale archives are closely integrated within the paral-
lel I/O environment through hierarchical storage management capabilities of
the parallel file system and the archival storage system allowing datasets to
automatically transition between these two distinct storage environments.
Part I of this topic details the parallel I/O environments of a number of
representative high performance computing facilities. While the specific tech-
nologies employed may differ from facility to facility, a great degree of com-
monality exists across these environments. In Chapter 2 the National Energy
Research Scientific Computing Center (NERSC) facility is detailed including
their use of multiple parallel file system environments for large-scale com-
pute platforms and a more general-purpose center-wide storage environment.
Chapter 3 provides an overview of the National Center for Supercomputing
Applications (NCSA), the first site in the United States to achieve over 1
Terabyte/sec of sustained parallel I/O bandwidth. The Argonne Leadership
Computing Facility (ALCF) is detailed in Chapter 4, unique in its use of PVFS
at such a significant scale. Chapter 5 provides an overview of the Livermore
Computing Center notable for their work to integrate the ZFS file system
with the Lustre parallel file system. The Los Alamos National Laboratory
HPC facility is highlighted in Chapter 6, an early proponent of center-wide
accessible parallel file systems and the incorporation of burst buffers within
the parallel I/O environment. Finally, we conclude Part I on HPC facilities
with an overview the Texas Advanced Computing Center, an early adopter of
commodity storage systems and software-based RAID within the parallel I/O
environment.
 
Search WWH ::




Custom Search