Hardware Reference
In-Depth Information
The HPSS data movers are dedicated Dell R720 servers with dual socket
Intel Sandybridge
TM
processors and 256 GB of memory. Each processor has
16 cores per socket. The data movers manage the movement of the data from
one storage level to another between network, disk, and tape. Each mover is
connected to eight tape drives, the disk cache fabric, and has two 40 GigE
network cards. The Dell 2U R720 system was chosen based on the number
of PCIe2 cards that could be placed within the system while maintaining full
performance to all attached devices and networks. The useable 1.2-PB caching
disk is used for the traditional archive data movement and data transfers. The
environment has two large core servers to manage the DB2
TM
database with
all the metadata.
The near-line storage subsystem provides over 300 usable petabytes of
storage to science and engineering teams with an aggregate bandwidth of
about 100 GB/s. The physical robotic units have a capacity of over 500 PB
if needed. The Blue Waters Project has an operating budget at current costs
to provide 380 raw petabytes of tape cartridges over the course of full service
operations. The implementation of will use about 22% of the raw capacity to
provide protection from single points of tape failure.
RAIT technology, developed by NCSA and the HPSS consortium as part
of the Blue Waters Project, is a new feature in that provides data integrity
and protection through striped data parity technologies. RAIT avoids the
traditional method of protection from tape errors that duplicates all user
data on two or more tapes (also known as RAID-1). With media costs in the
$16M range for 500 PB of data, an additional $16M cost for protection was
not feasible for most of the scientific data. RAIT provides a variety of data
and parity combinations from two data blocks plus one parity block written to
three tapes, up to eight data plus eight parity written 16 tapes wide, depending
on the data size and desired degree of redundancy for multiple failure. Data
is written in a RAID-5-like form with data and parity blocks rotated across
all tapes ensuring that the tapes are generally the same length and amount
of data written to them.
NCSA utilizes two data service classes utilizing RAIT. Large files are writ-
ten in a 4+1 RAIT configuration and the x-large storage class for files greater
than 8 TB are written directly to tape in a 7+2 RAIT configuration. The 7+2
utilizes the media cartridge holder capacity of the SpectraLogic library which
holds nine TS1140 tapes. A single mount request brings all nine tapes to the
tape drives in one robotic motion, which provides less latency for mounting
nine tapes before a write can begin.
Blue Waters intends to integrate the on-line and near-line storage subsys-
tems into a transparent hierarchy using new features in Lustre 2.5. This will
enable users to see a single storage repository namespace that transparently
moves the data they need between on-line and near-line subsystems so the
data is \in the right place at the right time." These features will enable more
ecient use of the storage resources and more science productivity.
