Hardware Reference
In-Depth Information
Chapter 8
Lustre
R
Eric Barton and Andreas Dilger
Intel
8.1
Motivation ::::::::::::::::::::::::::::::::::::::::::::::::::::::::
91
8.2
Design and Architecture :::::::::::::::::::::::::::::::::::::::::
92
8.2.1
Overview :::::::::::::::::::::::::::::::::::::::::::::::::
92
8.2.2
Networking ::::::::::::::::::::::::::::::::::::::::::::::
93
8.2.2.1
LNet ::::::::::::::::::::::::::::::::::::::::
93
8.2.2.2
RPC ::::::::::::::::::::::::::::::::::::::::
94
8.2.3
Distributed Lock Manager ::::::::::::::::::::::::::::::
96
8.2.4
Back-end Storage ::::::::::::::::::::::::::::::::::::::::
97
8.2.5
Metadata Server :::::::::::::::::::::::::::::::::::::::::
98
8.2.6
Object Storage Server :::::::::::::::::::::::::::::::::::
99
8.2.7
Management Server :::::::::::::::::::::::::::::::::::::: 100
8.2.8
Client :::::::::::::::::::::::::::::::::::::::::::::::::::: 100
8.2.9
Recovery ::::::::::::::::::::::::::::::::::::::::::::::::: 102
8.3
Deployment and Usage ::::::::::::::::::::::::::::::::::::::::::: 103
8.4
Conclusion :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 104
Bibliography :::::::::::::::::::::::::::::::::::::::::::::::::::::: 105
8.1 Motivation
The Lustre
1
file system is an Open Source (GPLv2) high performance
parallel file system widely used in the HPC community to provide a global
POSIX namespace and horizontally scalable I/O to the computing resources
of an entire datacenter. It was originally designed for scalability and is capable
of handling extremely large volumes of data and huge numbers of files with
high availability and strong coherence of both data and metadata. This makes
it the premier choice for the most demanding HPC applications spanning the
world's largest HPC clusters.
The Lustre file system originated in 1999 at Carnegie Mellon University
with a project led by Dr. Peter Braam to develop an object-based disk file
1
Other names and brands may be claimed as the property of others.
91
