Hardware Reference
In-Depth Information
TABLE 5.1: Comparison of some relevant performance-related characteristics
of the generations of Blue Gene systems that LC sited.
Feature
Blue Gene/L
Blue Gene/P
Blue Gene/Q
Cores per node
2
4
16+1+1
Clock speed
700 MHz
850 MHz
1.6 GHz
Memory per node
512 MB{1 GB
2-4 GB
16 GB
Memory bandwidth
5.6 GB/s
13.6 GB/s
43 GB/s
Peak performance
per node
5.6 GFLOPS
13.6 GFLOPS
204.8 GFLOPS
per node
Network
bandwidth
2.1 GB/s
5.1 GB/s
40 GB/s
Peak performance
410 TFLOPS
1 PFLOPS
20 PFLOPS
GFLOPS per watt
.23
.37
2.1
Cores/ION
128
256
2048
weak relative to the nodes on the commodity Linux systems, they would be
handling the I/O for all of the 64 compute nodes serviced by it.
The BG/P systems continued down the path forged by the BG/L de-
sign. There were more cores per compute node, but now some symmetric
multi-processing (SMP) operations were possible and there were more com-
pute nodes per I/O node.
Sequoia is a BG/Q system, following the BG/P systems. Again there are
faster processors with even more cores. There are relatively more compute
nodes per I/O node, leading to the situation that 2048 cores are now being
serviced by a single I/O node. The I/O node runs a full Linux operating
system, now based on a Red Hat Enterprise Linux release.
As with the initial BG/L system at LC, the first major challenge for the
local team of Lustre developers was to get the Lustre client working on the
BG/Q system. The Lustre client underwent a complete rewrite for version
2.0, and there was new software (now Red Hat Enterprise Linux-based) for
the I/O node. The implication of these changes was that getting client software
working on BG/Q put LC behind the desired schedule.
5.5 Sequoia File System Hardware
The file system hardware configuration for the Lustre file system for Se-
quoia is based on NetApp E5400 RAID devices with a Mellanox-based Infini-
Band SAN infrastructure. The OSS nodes were leveraged from the TLCC2 [11]
node design from the cost-effective computing platforms designed to support
