Hardware Reference
In-Depth Information
23.2 Design/Architecture
For in-transit processing to succeed, the I/O subsystem and software stack
must be designed to accommodate the execution of the analysis operations
to be performed on the in-flight data. These accommodations are built on
top of a base HPC system with burst buffers installed for checkpoint/restart
capability. In such a system, the compute nodes communicate with the parallel
file system by routing I/O requests over the high-speed fabric to an I/O node,
which routes the request onto a storage network where the storage resides. This
generalized setup can take many forms depending on the make and model of
the system in question. One potential setup is illustrated in Figure 23.1 The
burst buffer subsystem is installed either internal to or connected to the I/O
nodes of such a system, and takes the form of a pool of solid-state non-volatile
memory. This pool absorbs the bursts of I/O trac and through the I/O node,
has the ability the drain the content to disk for permanent storage, much like
a cache in a RAID controller buffers write requests to the attached disks.
However, when in-transit processing of data is desired, the burst buer's
configuration must be augmented to accommodate more than just buffering of
data in-flight to the parallel file system. In this scenario, the burst buffer flash
pool must be associated with directly attached compute capability, RAM, and
cross{burst buer communication links. Essentially, a place to execute the de-
sired analysis code is needed with the intent that it have high-bandwidth
FIGURE 23.1: Potential system diagram for a supercomputer with burst
buers.
 
Search WWH ::




Custom Search