Hardware Reference
In-Depth Information
and with HACC (see Section 18.3.1), a cosmology application, to capture its
particle data model. The GLEAN project has also worked with PHASTA [9],
an adaptive unstructured mesh data model, to support unstructured grids in
GLEAN. Additionally, pNetCDF and HDF5 APIs have been mapped to rele-
vant GLEAN APIs, thus enabling the developers to interface with simulations
using pNetCDF and HDF5. These represent the common HPC simulation data
models ranging from AMR grids to particles, and to unstructured adaptive
meshes.
18.2.3 Asynchronous Data Staging
Asynchronous data staging refers to moving the application's I/O data
to dedicated nodes, and next writing this data out to the file system asyn-
chronously while the application proceeds ahead with its computation. A key
distinguishing characteristic of GLEAN's data staging is that it leverages the
data models and semantics of applications for staging instead of viewing data
simply as files and/or buffers. On the staging nodes, GLEAN runs as an MPI
job and communicates with the GLEAN aggregator nodes using a network
abstraction layer customized to exploit the underlying interconnects. Asyn-
chronous data staging blocks the computation only for the duration of copying
data from the compute nodes to the staging nodes. The data staging serves as
a burst buffer for the simulation I/O that can be written out asynchronously to
the file system while the computation proceeds. Data staging also significantly
reduces the number of clients seen by the parallel file system, and thus miti-
gates the contention including locking overheads for the file system. The data
semantics enables GLEAN to transform the data on-the-fly to various I/O for-
mats. On the staging nodes, GLEAN aims to be deployed either as an \always
ON" service, run on dedicated set-aside nodes of a simulation, or co-scheduled
along with the simulation. If there is insucient memory on the staging nodes
(receivers), the transfer is blocked until sucient buffer/memory space is made
available. Staging can leverage local node storage to mitigate this issue.
18.2.4 Compression and Subfiling
To reduce the amount of data written to storage and thus improve the I/O
performance, GLEAN incorporates compression for I/O. The current imple-
mentation leverages Blosc [1], a blocking, shuing, and loss-less compression
library, for data compression and decompression; however, the API enables
programmers to invoke their own user-defined compression libraries to achieve
the best speed/compression ratio. To mitigate the overhead of the locks and
metadata of parallel le systems, GLEAN's I/O mechanism incorporates sub-
filing [4] wherein a few large files are written out to overcome the parallel
file system overheads of a single shared file and a file-per-process I/O. The
number of files written out is tunable. On the ALCF Blue Gene systems, files
are written out per I/O node.
