Hardware Reference
In-Depth Information
point to the same file location. The same filetype and its extent are applied
recursively to the file space, and all visible portions together comprise a pro-
cess's le view. The program shown in Figure 13.4(c) sets le views and calls
an independent write function to write the 2D array in parallel. The write
amount is of size (N 4) integers, equal to the entire subarray size in each
process. Note that the amount requested in an MPI-IO function call need not
be the same as the size of the filetype.
From this example, by dening a process's le view, writing the entire
subarray can be accomplished by making just one MPI-IO function call. This
program fragment also tells a different user I/O intent from the one expressed
in Figure 13.4(a). MPI-IO interprets the user's intent as to write the whole
subarray as a single request. The restriction of write order for the individual
subarray rows as intended by Figure 13.4(a) is eliminated. Instead of pro-
tecting data consistency for each individual row, the MPI-IO library enforces
the consistency for the entire subarray. Given such interpretation, an MPI-IO
implementation has more freedom to adopt certain optimizations to improve
performance. One of the well-known optimizations for such I/O requests is
called \data sieving" [12]. Since most of the modern le systems do not per-
form well for a large number of small, noncontiguous requests, data sieving
is an I/O strategy that makes large I/O requests to the file system and ex-
tracts, in memory, the data that is actually needed. Data sieving has been
incorporated into ROMIO, a very popular MPI-IO implementation developed
at Argonne National Laboratory [10]. When applied to the discussed example,
data sieving will first read a contiguous file segment that is large enough to
cover a process's entire le view, the whole subarray, into a temporary buer
and then copy the requested data to the user's buer. As a result, this read-
modify-write strategy reduces the number of I/O requests to the file system.
Significant performance improvement has been observed on noncontiguous
I/O requests when data sieving is enabled.
13.3.3 Collective I/O
The MPI file view feature gets us one step closer to meeting the user
I/O intent for this example, which is to write a global 2D array in parallel
from four processes as a single I/O operation. However, there are still four
independent I/O requests, one from each process. Hence, the MPI-IO library
must protect the data integrity for the four requests individually. When data
sieving is enabled, the entire file region involved in a single read-modify-write
operation must be protected from another. A common solution to provide
such protection is to use an advisory file locking mechanism to guarantee the
exclusive access for a file region from concurrent access. In this example, the
region to be locked by process rank 0's request overlaps with rank 1's lock
region. Similarly, process rank 1's lock region overlaps with the one requested
by rank 2. As a result, only half of the four processes are actively writing their
data to the file system while the other half are waiting for their lock requests
