Database Reference
In-Depth Information
The design and implementation of DataTap servers deal with several per-
formance issues and constraints present on modern MPP machines. First, due
to the limited amount of memory available on the DataTap server, the server
only issues a read call if there is memory available to complete it. Second,
since buffer space used by asynchronous I/O on compute nodes is limited,
the server issues multiple read calls each time it operates. Third, the next
generation of DataTap servers will install controls on the speed and timing
of reading data from DataTap buffers. The goal is to prevent perturbation
caused when I/O actions are performed simultaneously with internal commu-
nications of application code (e.g., MPI collectives). Additional constraints
result from in-transit actions performed by I/O graphs; these are evaluated
in our ongoing and future work.
The current implementation of DataTap leverages existing protocols (i.e.,
Cray Portals and InfiniBand RDMA). Since the abstraction presented to the
programmer is inherently asynchronous and data driven, data movement can
take advantage of data object optimizations like message aggregation, data
filtering, or other types of in-transit data manipulations, such as data val-
idation. In contrast, the successful paradigm of MPI-IO, particularly when
coupled with a parallel file system, heavily leverages the file nature of the
data target and utilizes the transport infrastructure as eciently as possible
within that model. That inherently means the underlying file system con-
cepts of consistency, global naming, and access patterns will be enforced at
higher levels as well. By adopting a model that allows for the embedding of
computations within the transport overlay, it is possible to delay execution
of or entirely eliminate those elements of the file object that the application
does not immediately require. If a particular algorithm does not require con-
sistency (as is true of some highly fault-tolerant algorithms), then it is not
necessary to enforce it from the application perspective. Similarly, if there is
an application-specific concept of consistency (such as validating a checkpoint
file before allowing it to overwrite the previous checkpoint file), then that
could be enforced, as well as all of the more application-driven specifications
mentioned earlier.
DataTaps leverage extensive prior work with high-performance data move-
ment, including (1) ecient representations of meta-information about data
structure and layout using a Portable Binary I/O format (PBIO
16
); which en-
ables (2) high performance and “structure-aware” manipulations on data in
flight, carried out by dynamically deployed binary codes and using higher level
tools with which such manipulations can be specified, termed XChange.
17
; (3)
a dynamic overlay (i.e., the I/O graph) optimized for ecient data move-
ment, where data fast path actions are strongly separated from the control
actions necessary to build, configure, and maintain the overlay
*18
; and (4)
a lightweight object storage facility (LWFS
19
) that provides flexible, high-
performance data storage while preserving access controls on data.
*
Such control actions are referred to as the control layer for the overlay network.
