Hardware Reference
In-Depth Information
by utilizing low-power consumption processor technology, but by using this
technology at a greater scale to achieve higher levels of performance.
LC has used the Lustre file system on all three generations of Blue Gene
systems. Sequoia is the third-generation BG/Q system [2], and went into pro-
duction at Livermore in 2013.
5.2 The Lustre
R
Parallel File System: Early Develop-
ments
Lustre [7, 8, 12] is a file system architecture and implementation suitable
for very large clusters, such as those at LC. Lustre was initially designed,
developed, and maintained by Cluster File Systems, Inc. (CFS) within an
ASCI Path Forward project funded by the DOE. The initial collaboration
with CFS included Hewlett{Packard and Intel and the three NNSA research
laboratories: Livermore, Los Alamos, and Sandia.
Sun Microsystems acquired CFS in 2007 to fortify their storage business.
After their acquisition of Sun in 2010, Oracle mapped out their exit from
Lustre involvement. WhamCloud, Inc. was formed in 2012 as an entity to
provide support and development independently from Oracle, only later to be
acquired by Intel. Oracle eventually sold their remaining rights interests to
Xyratex, Ltd. in 2013.
Lustre runs on commodity hardware, using object-based storage and sepa-
rate metadata servers to isolate these functions and improve scalability. A key
advantage of OSDs in a high performance environment is the ability to dele-
gate low-level allocation and synchronization for a given segment of data to
the device on which it is stored, leaving the file system to decide only on which
OSD a given segment should be placed. Since this decision is quite simple and
allows massive parallelism of bulk data transfer, each OSD need only man-
age concurrency locally, allowing a file system built from thousands of OSDs
to achieve massively parallel data transfers. Lustre is a POSIX-compliant file
system presenting a unified client interface such as
open()
,
read()
,
write()
,
etc. to the application. This is the portion of the file system that is running on
(or close in the case of the Blue Gene systems) to the node doing the computa-
tion. At LLNL, Lustre uses the InfiniBand network switch in the more recent
instances. Network independence is another Lustre design strength, facilitated
originally by its use of the Portals protocol stack, an abstract approach to net-
working originally developed at Sandia National Laboratory (now available as
open source software). This functionality, now called LNet, is implemented as
part of Lustre at LLNL.
