Hardware Reference
In-Depth Information
the augmented HDF5 library takes advantage of these to ensure consistent
transitions of a given dataset.
Another interesting feature of the Intel Fast Forward work is the manner
in which burst buffers are integrated into the overall system. I/O nodes that
manage local NVRAM export three distinct data models for use by application
libraries: a blob object (i.e., a byte stream), a keyword-value object, and an
array object. All are used within HDF5, and the I/O node is responsible for
mapping transactions on these constructs into operations on DAOS objects.
This issue of translations between abstractions at various levels of the I/O
stack is critical, and often overlooked.
30.3.3 Namespaces
Despite hierarchical file systems being declared as dead [17], this model of
data organization still dominates in HPC. While both the EOF and Intel Fast
Forward projects provide alternative organizations for data, neither makes
radical changes to the way we organize and reference named entities.
What seems most likely is that the role of providing namespaces for HPC
applications will be taken over by upper layers of the HPC storage stack, layers
such as HDF5, PnetCDF, and others, with storage systems instead providing a
richer building-block abstraction on which these namespaces are constructed.
What is even less clear is how search-like capabilities will be integrated into
the HPC storage stack. Activities such as hFAD [17] and Spyglass [14] have
provided some insight, but ultimately HPC search is about application data
structures | not something that the low-level storage system is necessarily
even aware of. Perhaps active storage advances will be what ultimately enable
application-oriented searches to be executed local to storage where they can
be performed eciently.
30.4 Conclusion
The POSIX storage model has been a cornerstone of HPC systems for
decades. As a result of its success, HPC system providers have been slow to
adopt alternatives to the well-understood POSIX file model, but the increasing
use of I/O libraries such as HDF5 and PnetCDF by scientific codes have
eliminated many long-standing dependencies on the antiquated POSIX model
and provided new ways for science teams to interact with storage.
Research has identified a number of promising alternative models, and
the community seems ready to embrace a new model. New additions in the
storage hierarchy, such as the inclusion of nonvolatile memory in the storage
system, have further disrupted the status quo. With current storage systems
struggling to keep pace, the timing could not be better.
