Database Reference
In-Depth Information
also ensures that key data analysis capabilities are present during the earliest
phases of the software development effort. The H5Part file format and APIs
enable disparate research groups with different simulation implementations to
transparently share datasets and data analysis tools. For instance, the com-
mon file format enables groups that depend on completely different simulation
implementations to share data analysis tools.
H5Part is built on top of HDF5 (hierarchical data format). HDF5 offers
a number of advantages: It is a self-describing, machine-independent binary
file format that supports scalable parallel I/O performance for MPI codes
on a variety of supercomputing systems and works equally well on laptop
computers. HDF5 is available for C, C++, and Fortran codes. The primary
disadvantage of HDF5 is in the complexity of the API. Because of the rich set
of functionality in HDF5, it can be challenging for domain scientists to write
out a simple 1D array of data using raw HDF5 calls. Worse, doing parallel I/O
is further complicated by the variety of data layout and I/O tuning options
available in HDF5.
By restricting the usage scenario to particle accelerator data, H5Part en-
capsulates much of the complexity of HDF5 to present a simple interface for
data I/O to accelerator scientists that is much easier to use than the HDF5
API. Compared with code that calls HDF5 directly, code that calls H5Part
is more terse, less complex, and easier to maintain. For example, code that
uses H5Part needs to make any HDF5 calls that set up organization for data
groups inside the HDF5 file since it encapsulates such functionality. The in-
ternal layout of data groups inside the HDF5/H5Part file has proven to be
effective for both eciently writing data from simulation code as well as for
eciently reading data, either serially or in parallel, into visual data analysis
tools.
9.4.1.2 File Organization and API
The H5Part file storage format uses HDF5 for the low-level file storage and a
simple API to provide a high-level interface to that file format. A programmer
can use the H5Part API to access the data files or to write directly to the file
format using some simple conventions for organizing and naming the objects
stored in the file.
In order to store particle data in the HDF5 file format, we have formal-
ized the hierarchical arrangement of the datasets and naming conventions for
the groups and associated datasets. The H5Part API formally encodes these
conventions in order to provide a simple and uniform way to access these
files from C, C++, and Fortran codes. The API makes it easy to write very
portable data adaptors for visualization tools in order to expand the number
of tools available to access the data. Users may write their own HDF5-based
interface for reading and writing the file format, or may use the h 5 ls and
h 5 dump command-line utilities, which are included with the HDF5 distribu-
tion, to display the organization and contents of H5Part files. The standards
Search WWH ::




Custom Search