Hardware Reference
In-Depth Information
TABLE 6.1: Detailed summary of LANL's 2013 computing environment.
Total machines
20
Nodes
18321
Cores/cell units
354232
Memory
637472 GB
Largest machine
107,000 cores
Memory
212 TB
Size of largest machine
1.37 PFLOP/s
Installation
2010
Size of future largest machine
50{100 PFLOP/s
Memory
2{4 PB
Installation
2015{2016
6.2.1 Storage Environment
The SANs at LANL use 10 GigE and IB-based interconnects. The file
systems are Panasas- and Lustre-based. The parallel archives in use are the
HPSS and IBM GPFS with Tivoli storage managers (TSM). Both archive
systems are hierarchical storage management systems with parallel tape as
the bulk storage method.
6.2.2 Storage Area Networks
LANL utilizes the typical HPC topology of compute nodes connected to
I/O nodes via the cluster interconnect. The I/O nodes are connected to the
parallel scratch file system via a scalable SAN. Additionally, non-scalable
services like NFS for home and project spaces, and parallel archives are con-
nected to the scalable SAN.
From 2001 to 2002, LANL deployed its first large Linux cluster, which
contains more than 1,000 nodes. One of the main requirements in deploying
TABLE 6.2: Summary of the LANL storage environment categorized by at-
tributes, largest environment, largest machine, and the total available site-
wide.
Attribute
Largest Env
Largest Mach
Total
SAN Bandwidth TiBits/s
34
6
40
SAN Ports GigE/IB QDR
2688/192
0/192
2920/192
I/O Nodes
666
192
772
Scratch Storage PB
20
10
23
Scratch Storage GB/s
346
160
409
Home and Project Space TB
800
Shared
1000
File Systems
8
3
11
