Hardware Reference
In-Depth Information
FIGURE 15.2: The NetCDF file layout (used in both serial NetCDF and
Parallel-NetCDF). A small header describes the contents of the file and any
attributes on variables, dimensions, or the file itself. Variables follow in con-
tiguous regions. The \non-record variables," those that can grow in one di-
mension, follow in an interleaved fashion.
established file format precluded parallel I/O, so Parallel-NetCDF used that
file format as the foundation. By maintaining file format compatibility, scien-
tists could introduce Parallel-NetCDF to their workflows with minimal dis-
ruption. Parallel-NetCDF could not implement some of its ideas for optimized
parallel I/O without altering the programming API, but such alterations were
done in the spirit of serial-NetCDF, and retain much of the same feel and
semantics.
Both libraries support the \classic" netCDF le format, depicted in Fig-
ure 15.2. The first bytes of the file contain a header. This header describes the
name and size of the dimensions used by the variables; the name, type, shape,
and starting location in the le of variables; and annotations or \attributes"
associated with dimensions, variables, or the entire dataset.
Serial netCDF introduced the idea of a bi-modal API: to write a dataset,
a program enters \dene mode." In dene mode, the programmer denes the
dimensions being used, associates those dimensions with variables, possibly
annotates the components of the dataset, and then switches to \data mode."
The program can only transfer data after it has described how the data will be
stored. This pre-declaration of the le structure benets parallel I/O greatly|
a process can compute where every element of every array will go, and need
not coordinate with any other process. The listing in Figure 15.3 provides a
brief example of how define mode and data mode work together. Readers can
find entire code samples in the on-line Parallel-Netcdf Quick Tutorial [1].
15.4 Deployment and Usage
In many situations, the choice of which I/O library to use depends on
the domain in which one operates. Climate codes, for example, have a large
repository of datasets in the classic netCDF file format. Analysis and visual-
ization tools were written to read this file format, so when a climate simulation
generates data, that data should be in netCDF format to simplify data man-
agement.
Parallel-NetCDF, despite a different API and its parallel I/O features,
maintains compatibility with the serial netCDF file format. This compatibility
