Hardware Reference
In-Depth Information
FIGURE 8.1 (See color insert): Lustre architecture. [Image courtesy of Intel
Corporation.]
as Infiniband or TCP/IP are abstracted by the Lustre Networking layer, LNet.
LNet provides both message passing and remote memory access (RMA) for
ecient zero-copy bulk data movement. The Lustre RPC layer (Ptlrpc) is
built on top of this to provide robust client{server communications in the face
of message loss and server failures.
The Lustre Distributed Lock Manager (LDLM) is a service provided by
storage targets in addition to object storage services. LDLM locks are used to
serialize conflicting file system operations on objects managed by that target,
and are the mechanism used to ensure distributed cache coherency. The com-
bination of coherent locking and recovery protocols exercised on server startup
ensure that caches remain consistent through server restart or failover. This
boosts server throughput for file system modifying operations by allowing the
use of write-back rather than write-through caches for file system-modifying
operations since uncommitted operations are recovered from the clients in case
of server failure.
8.2.2
Networking
8.2.2.1
LNet
Lustre networking (LNet) was originally based on a port of Sandia Por-
tals [5] to the Linux kernel. Portals has a very clean and simple API which
supports non-blocking network operations and both dual-sided (message pass-
ing) and single-sided (RMA) styles of communication. Together these made
Portals ideal for the implementation of Lustre RPC since non-blocking opera-
