Hardware Reference
In-Depth Information
some cases since there is junk data that must be read in along with requested
data. In future storage systems with SSDs (solid-state drive) where disk seek
is inexpensive, this staged read approach might not be the best solution and
this is left for future study.
22.3 Conclusion
S3D offers a challenging I/O problem at scale, a problem that ADIOS
is able to tackle in a compelling and usable manner. The obstacles faced in
scaling up I/O performance for S3D are general obstacles that many applica-
tions will face. By combining aggregation, subfiling, and chunking techniques,
ADIOS provides a solution that can be generalized for many I/O problems.
The comparison of the N-M I/O pattern to N-1 and N-N patterns provides a
higher-level view of the file system limitations in current platforms.
The key advantages of staged I/O are three-folded. First, it effectively
aggregates small I/O accesses into large chunks and hence eliminates disk
seek operations. By allowing only a subset of MPI processors to perform I/O
operations to the storage system, it further avoids lock contention for writing.
On the other hand, it significantly reduces the cost of metadata operations
such file open and close for reading and memory consumption. This scheme
is implemented as a stand-alone I/O method inside the ADIOS framework
so that existing ADIOS applications can benefit from it without source code
change.
This approach underlies the philosophy of the ADIOS platform. The
service-based approach of ADIOS allows the development of new techniques
and methods that can be easily shared with the I/O community at large, pro-
viding new insights into I/O performance for a vast variety of applications.
Bibliography
[1] Cray XE6 Brochure. http://www.cray.com/Assets/PDF/products/xe/
CrayXE6mBrochure.pdf .
[2] J. Chen, A. Choudhary, B. de Supinski, M. DeVries, E.R. Hawkes,
S. Klasky, W. K. Liao, K. L. Ma, J. Mellor-Crummey, N. Podhorszki,
R. Sankaran, S. Shende, and C. S. Yoo. Terascale Direct Numerical Simu-
lations of Turbulent Combustion Using S3D. Computational Science and
Discovery, 2(1):015001, 2009.
 
Search WWH ::




Custom Search