Database Reference
In-Depth Information
scientist creates an XML file that contains this information, along with the
specification of the method for each group, such as message passing inter-
face input/output (MPI-IO), or portable operating system interface for unix
(POSIX). The method declarations can be switched at runtime and allow the
scientist to change from POSIX I/O, to MPI-IO, 8 to asynchronous methods
such as the DataTap services 9 and the DART system 10 described below. By
allowing the scientist to separate out the I/O implementation from the API,
users are allowed to keep their code the same and only change the underly-
ing I/O method when they run on different computers. Another advantage of
specifying the information in this manner is that the scientist can just main-
tain one write statement for all of the variables in a group, thus simplifying
their programs. This system also allows the user to move away from individual
write statements, and as a result, the system can buffer the data and conse-
quently write large blocks of data, which works best in parallel file systems.
A small example of an XML file is as follows.
<
ioconfig
>
< datatype name=''restart'' >
< scalar name=''mi'' path=''/param'' type=''integer''/ >
< dataset name=''zion'' type=''real'' dimensions=''n,1:4,2,mi''/ >
< data
attribute name=''units'' path=''/param'' value=''m/s''/ >
< data
attribute name=''long
{\ }
name'' path=''/param'' value=''ion
parameters''/ >
</
>
datatype
< method priority=''1'' method=''DATATAP'' iterations=''1'' type=''diagnosis'' >
srv=ewok001.ccs.ornl.gov <
/
method >
>
Most importantly, however, ADIOS can provide such methods with rich
metadata about the data being moved, thereby enabling the new paradigms
for high-throughput data movement. These new paradigms include: (1) com-
pact binary data transmission using structure information about the data
(e.g., for ecient interpretation of data layout and access to and manipula-
tion of select data fields): (2) the ability to operate on data as it is being
moved (e.g., for online data filtering or data routing); and (3) the ability
to use appropriate underlying transport mechanisms (e.g., such as switching
from MPI-I/O to POSIX, to netCDF, to HDF-5). Furthermore, we envision
building a code-coupling framework extending the ADIOS APIs that will
</
ioconfig
Search WWH ::




Custom Search