Database Reference
In-Depth Information
In-situ visualization clients
Forward to in-situ
visualization
clients
Compute Nodes
Forward particles
to storage services
Separate storage
for particle-based
and timestep-
based formats
Figure 5.1
I/O graph example.
Scheduling techniques dynamically manage DataTap and I/O graph
execution, taking into account the I/O costs imposed on petascale
applications.
Experimental results attained with DataTaps and I/O graphs demon-
strate several important attributes of I/O systems that benefit petascale
machines. First, asynchronous I/O makes it possible to carry out I/O ac-
tions while massively parallel processor (MPP) computations are ongo-
ing. This computation-I/O overlap improves throughput substantially,
compared with the synchronous methods used by current file systems.
Second, when performing I/O asynchronously, we demonstrated that
it can scale without perturbing the applications running on compute
nodes. For instance, sustained high-bandwidth data extraction (over
900 MB/s) has been achieved on the Cray XT4 without undue applica-
tion perturbation and with moderate buffering requirements. 9
DataTaps Implementation A DataTap is a request-read service designed
to address the difference between the available memory on typical MPP com-
pute partitions and that on I/O and service nodes. We assume the existence
of a large number of compute nodes producing data—DataTap clients—and
a smaller number of I/O nodes receiving the data—DataTap servers. The
DataTap client issues a data-available request to the DataTap server, encodes
the data for transmission, and registers this buffer with the transport for
remote read. For very large data sizes, the cost of encoding data can be sig-
nificant, but it will be dwarfed by the actual cost of the data transfer. 13 15
On receipt of the request, the DataTap server issues a read call. The DataTap
server feeds an I/O graph, which can replicate the functionality of writing the
output to a file, or it can be used to perform “in-flight” data transformations.
Search WWH ::




Custom Search