Hardware Reference
In-Depth Information
1
/*posttwonon-blockingoperations,writingdatato
2
twovariables(varid1,varid2)inthe\dataset{}*/
3
ncmpi_iput_vara(ncfile,varid1,&start,&count,&data,count,
4
MPI_INT,&requests[0]);
5
ncmpi_iput_vara(ncfile,varid2,&start,&count,&data,count,
6
MPI_INT,&requests[1]);
7
8
/*hereinncmpi_wait_allthelibrarywillinspectallnon-blocking
9
operations,combinethem,andservicethenewlargerrequestcollectively*/
10
ncmpi_wait_all(ncfile,2,requests,statuses);
11
if(ret!=NC_NOERR)handle_error(ret,__LINE__);
FIGURE 15.4: A code fragment demonstrating the use of the non-blocking
routines. The Parallel-NetCDF library will stitch these two requests into one
single more ecient MPI-IO operation.
data transfer methods in serial netCDF (the var, vars, vara, varm routines) all
take a contiguous memory buer. Parallel-NetCDF's Flexible Interface allows
the caller to specify the type and structure of memory with an MPI datatype.
Jianwe Li's SC 2003 paper [4] goes into more detail.
Parallel-NetCDF also introduced non-blocking operations in its API. These
nonblocking operations were not interesting at first: they would call the non-
blocking MPI-IO routines, but most MPI-IO implementations provided little
if any non-blocking support. The non-blocking interface, however, provided a
good location to apply an operation-combining optimization.
Figure 15.4 lists a code fragment using this non-blocking interface. Parallel-
NetCDF's non-blocking interface follows the same post-and-wait model used
in MPI.
Both the Flexible Interface and the non-blocking interface re-enforce a
common theme in high performance parallel I/O: providing as much informa-
tion as possible to the storage system. In Latham et al.'s case study [3], using
these extended features of Parallel-NetCDF improved checkpoint bandwidth
at scale by a factor of 3.
15.6 Conclusion
The I/O library has seen much research in the decade since Parallel-
NetCDF began. HDF5 continues development. ADIOS offers an alternative
approach to data management. The serial netCDF project has incorporated
parallel I/O features by implementing its API (with some extensions) on top
of the HDF5 library. Despite these innovations, Parallel-NetCDF still provides
a useful tool in the toolbox of parallel I/O libraries.
The first challenge a scientific application faces when managing data is
how to drive the large storage systems in parallel. The developers must first
