Hardware Reference
In-Depth Information
Chapter 14
PLFS: Software-Defined Storage for
HPC
John Bent
EMC
14.1
Motivation :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 169
14.2
Design/Architecture :::::::::::::::::::::::::::::::::::::::::::::: 170
14.2.1
PLFS Shared File Mode ::::::::::::::::::::::::::::::::: 170
14.2.2
PLFS Flat File Mode :::::::::::::::::::::::::::::::::::: 172
14.2.3
PLFS Small File Mode :::::::::::::::::::::::::::::::::: 172
14.3
Deployment, Usage, and Applications ::::::::::::::::::::::::::: 173
14.3.1
Burst Buers :::::::::::::::::::::::::::::::::::::::::::: 174
14.3.2
Cloud File Systems for HPC :::::::::::::::::::::::::::: 174
14.4
Conclusion :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 175
Bibliography :::::::::::::::::::::::::::::::::::::::::::::::::::::: 176
14.1 Motivation
Just as virtualization has provided massive flexibility for computation
across diverse processor hardware, software-defined storage provides that flex-
ibility for I/O workloads across diverse storage hardware. The Parallel Log-
structured File System (PLFS) is one such software-defined storage platform.
Originally designed as middleware to solve massively concurrent checkpoint-
ing, PLFS has become a powerful reminder of how not all workloads are well
suited to all storage systems. PLFS is also a compelling example of how a
layer of software indirection can change the base abilities of existing storage
systems. PLFS has since extended its functionality to address additional stor-
age challenges, thereby growing from single-purpose middleware into a more
general software-defined storage platform.
The main benefit of PLFS is that a system administrator can buy a single
storage system, configure it just once, and then use PLFS to allow a variety
of workloads to use that single storage system. The basic mechanism of PLFS
is an interception and transformation of unstructured application I/O into
well-structured I/O better suited to the underlying storage system. PLFS has
169
 
Search WWH ::




Custom Search