Hardware Reference
In-Depth Information
Acceptance testing of the Blue Waters near-line system stressed data sizes,
volumes, and rates projected to occur during the third and fifth year of the
project to verify the system would meet its fifth year requirements and goals.
The NCSA HPSS near-line system achieved on average 3,538 creates per sec-
ond (212,280 transactions per minute), which exceeds Blue Waters projected
year 6 workload. Blue Waters' nal test was to put the system under a 24-
hour system load test. During normal operations of media repacks and other
normal operations tasks, the near-line system wrote 9.8 million files and read
3.9 million files, and recalled more than 191,000 files from tape using all five
storage classes from extremely small files to extremely large files. This scale
and performance is well beyond any other system as of 2014.
Dedicated server systems, called Import/Export (IE) servers in Blue Wa-
ters (others label similar systems data transfer nodes or DTNs), move data
into and out of the on-line disk subsystem, between the on-line disk envi-
ronment and the near-line and onto the local-area and wide-area networks
to other resources. The 28 Import/Export servers (IE servers) are the same
Dell 720 configuration as the data movers, but have 192 GB of memory and
different PCIe cards. With the 40 GigE LAN environment, data can be trans-
ferred to the near-line system described above or to the local-area or wide-area
network environment to be stored elsewhere. Acceptance and periodic regres-
sion testing show sustained performance of 4 GB/s for each IE server for an
aggregate sustained I/O bandwidth of 112 GB/s.
Blue Waters uses the.
8
Globus software suite for data transfers between
the on-line storage subsystem and either the near-line subsystem or other
sites across the wide-area network connections. The main advantages of the
Globus environment are the \drag and drop" user interface, a ight control
interface enabling users to see transfers as they happen, and the retry feature
that automatically restarts interrupted transfers. NCSA encourages users to
checksum their transfers. Checksums are performed at both ends of the data
transfer to ensure data integrity. Checksumming adds to the transfer time and
processing capacity at both ends, but ensures that the user transaction has
completed properly.
The combination of the dedicated Lustre IE servers and the separate, but
also dedicated near-line data movers, provide a balanced and stable data man-
agement strategy to data movement. The ability to tune IE servers for striped
data transfers into and out of Lustre has proven to perform better depend-
ing on the size of the data being transferred. Tuning not only the GridFTP
client on the IE servers, but the Lustre client version and all its interdepen-
dencies including the IB fabric driver separate from other services, has made
the administration of the data movement environment much simpler by com-
bining services into the same servers. NCSA is working to bring in a striped
Globus server in 2014 for the on-line and near-line data transfer environment.
This development work will enhance the single data transfer rates. Along with
8
https://www.globus.org/
