Hardware Reference
In-Depth Information
5.4 IBM Blue Gene Systems
The IBM Blue Gene [3, 4] Supercomputer system design began in a re-
search and development effort at IBM with the combined objectives of study-
ing protein folding and investigating novel ideas in massively parallel machine
architecture and software. The BG/L system that followed was designed as a
more general-purpose supercomputer that combined those ideas with a net-
work where messages could be routed from any node to any other and a
parallel I/O subsystem.
Some of the unique aspects of the BG/L system were: processor speed
reduced for lower power consumption; scaling architecture leading to a large
range of system configurations; toroidal interconnect plus auxiliary networks
for global communication, I/O and system management; lightweight operating
system on compute nodes for minimal system overhead and minimal operat-
ing system \noise"; I/O functions shipped to I/O nodes, which run a more
comprehensive operating system; and 64 compute nodes per I/O network for
the installation.
One of the unique and interesting aspects of the Blue Gene computer sys-
tems is the particular packaging of the hardware components (here called the
\Blue Gene Scaling Architecture") into a hierarchy, allowing the conguration
of systems over a great range of scale. All of the generations of Blue Gene sys-
tems have followed this design concept. To give a bit more detail, let's look at
each level of the hierarchy while traversing a BG/Q system upward from the
CPU chip to a full Sequoia system [13]. In terms of the hardware components,
there is (ascending the hierarchy) (1) the chip with 16 cores, 16 quad-FPUs,
memory controllers, networks, etc. on a single chip; (2) a single compute (or
I/O) card, which has one chip with some memory available; (3) a drawer with
32 compute cards comprising a \drawer-like" node card, and eight I/O cards
that comprise an I/O drawer; (4) a midplane with 16 node cards; (5) a rack
which consists of two midplanes and an I/O drawer; and finally, (6) a system
composed of multiple racks that connect to complete the system (i.e., in Se-
quoia's case there are 96 racks in the system). This hierarchy is represented
pictorially in Figure 5.1.
For the purposes of discussing I/O on BG/Q systems, there are many
similarities and some differences between the compute node and I/O node.
Note that both are at the same level of the hardware packaging hierarchy in
Figure 5.1. Some similarities include that the compute nodes and I/O nodes
are physically set up to be virtually identical; and both are composed of the
same components (one BG/Q compute chip; 72 SDRAM chips; and connectors
to the power, JTAG, and 5D Torus network).
However, the main differences are due to how the nodes are used, or what
their function is meant to be. Compute nodes run a light-weight, Linux-based
kernel, called the compute node kernel (CNK), whereas the I/O nodes run
