Hardware Reference
In-Depth Information
access to the stored data in the burst buer. The cross{burst buer communi-
cation allows for the analysis application running to perform global operations
across all data stored within a set of storage pools. An example of this would
be computing the average ocean temperature across all stored grid points
stored in all burst buffers assigned to the job.
In addition to the hardware changes, software capability is also required
to make in-transit data analysis a success. First, a standard API for invoking
analysis operations in the burst buffer subsystem is needed. This API should
be simulation agnostic and serve as a piece of middleware that each appli-
cation leverages. Second, a standardized way for simulation applications to
communicate the current status of their I/O operations to the burst buffer
processing elements is needed. This communication not only serves to notify
the burst buffers as to what data is being sent down (which timestep, which
variables, etc.) but also to alert the analysis application as to when all of the
data is present on the burst buffers and available for operation. This commu-
nication could take the form of either ags in the le's metadata or explicit
communication over the network.
23.3 Systems Prototypes Related to Burst Buffers
One initial prototype that sets the stage for using burst buffers in modern
HPC systems is called \Overlapped Checkpointing with Hardware Assist" [6].
In the overlapped checkpointing system, the I/O nodes in an HPC system
are configured with a RAM disk or some type of fast non-volatile storage
(what is now flash in current implementations) running PVFS and mounted
onto the compute nodes. Applications wishing to checkpoint are linked with a
shim header, which intercepts MPI-IO's open and close commands to redirect
the write path from the parallel file system to the storage on the I/O nodes.
Daemons on the I/O nodes in turn handle the drain of the data from the
local storage pool to the actual expected path on the parallel file system.
This concept allows applications to write data to storage quickly and return
to computation while the storage system handles the data's safe transfer to
disk asynchronously. This idea has morphed into the current idea for a burst
buffer as outlined in studies by Grider et al. and Liu et al. [4, 5] where flash
pools are connected to the I/O nodes and software integration is much more
rened.
Argonne National Laboratory's GLEAN system [7, 8] (also see Chapter 18)
is a framework designed to provide in situ and in-transit storage and pro-
cessing services. Using GLEAN, application data can be migrated to staging
nodes within the HPC system and processed in-transit, all while maintaining
the application's data structure semantics. This has allowed, for example, one
demonstration of GLEAN where the PHASTA computational fluid dynamics
