Hardware Reference
In-Depth Information
Chapter 27
Darshan
Philip Carns
Argonne National Laboratory
27.1
Features :::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 309
27.2
Success Stories ::::::::::::::::::::::::::::::::::::::::::::::::::: 311
27.3
Conclusion :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 313
Bibliography :::::::::::::::::::::::::::::::::::::::::::::::::::::: 314
Darshan is an application-level I/O characterization tool that captures
production-level I/O behavior with minimal overhead. Darshan does not
record a complete trace of all I/O system calls. It instead gathers compact ac-
cess pattern statistics for each file opened by the application. These statistics
are reduced, compressed, and aggregated into a single log file that summarizes
the I/O activity and access patterns of the application as a whole. Although
this summary data does not offer the same fidelity as a traditional tracing or
profiling tool, it can be collected with negligible overhead and no source code
modification. This combination of features makes it possible not only to in-
strument full-scale application runs, but also to transparently deploy Darshan
for the automatic characterization of all production jobs on a leadership-class
HPC system. Darshan characterization data can be used for a variety of pur-
poses ranging from performance tuning of specific applications [6, 7, 8] to
analysis of trends in system-wide I/O behavior [1, 2].
Although Darshan is designed for system-wide deployment, it can also be
installed and used by individual end users as well. The runtime component
of Darshan consists of a set of user-space libraries and compiler wrappers
to simplify development environment integration. The command line utility
component of Darshan includes tools to interpret application logs and produce
graphical summaries for high-level analysis.
27.1 Features
The initial motivation for the Darshan project was to gain a better un-
derstanding of production I/O behavior by performing system-wide workload
309
