Hardware Reference
In-Depth Information
Data Moved for /intrepid-fs0
Read for /intrepid-fs0
4500
read
4000
3500
3000
2500
2000
1500
1000
500
0
Date
Write for /intrepid-fs0
4500
write
4000
3500
3000
2500
2000
1500
1000
500
0
Date
FIGURE4.4:Theamountofdatatransferredon
/intrepid-fs0
in 1-minute
intervals from March 2010 through July 2013.
that I/O is bursty, which is a long-held belief of HPC I/O and which this
data supports; and (2) read is quite important. Many people view HPC I/O
as mainly a write-intensive workload and read being secondary, but this data
indicates that read performance is equally as important to ALCF users as
write performance.
Another aspect of file system utilization is how applications store files,
which is based on how many and what size. Figure 4.5 shows the cumula-
tive distribution of file sizes on
/intrepid-fs0
. The ALCF discovered that
users create large numbers of files. The number of files on
/intrepid-fs0
is 402,687,835 for July 2012. The typical workload responsible for generat-
ing such a large number of files is using a file-per-process technique. Using a
file-per-process also results in generally smaller files, where small is relative to
the size of the file system. The design of the file system is primarily designed
