Hardware Reference
In-Depth Information
plain disks. OSDs differ in that they typically offer byte access granularity (as
opposed to sector granularity). The OSD implements the storage operations
(data transfer and layout), without defining any policy.
Many parallel file system implementations today, while exporting a mostly
POSIX-compliant model to their clients, internally access local storage using
an object storage model similar to the T10 standard [6, 7, 19, 18]. Unfortu-
nately, as seen with MPI-IO, these object storage abstractions are frequently
built on top of an existing local POSIX file system. This means potential e-
ciency gains (for example, in avoiding directory overhead) are often not fully
realized. At least one reason for this insistence on building on top of POSIX
is that no other data storage model comes close in terms of availability and
portability.
Luckily, some alternatives are now starting to appear. Seagate, a well-
known American data storage company, recently commercialized a disk drive
which exposes storage only as object storage API, as opposed to exposing low-
level sectors, as is still common for disk drives. A widely adopted standardized
object-based access method for low-level storage could finally provide libraries
and applications an equivalent solution providing the portability of POSIX
with the flexibility to define their own access semantics (consistency) and
grouping structures (directories or alternatives).
30.3 Post POSIX
Currently, the POSIX model for storage still dominates in HPC. However,
there is increasing use of libraries such as PnetCDF and HDF5 that provide
alternative data models to users. These libraries create an opportunity for
storage system designers: new underlying storage models can be deployed by
mapping these libraries directly onto the new storage model, avoiding the need
to support the POSIX model at all.
Thus, the HPC community may be at the cusp of a \post-POSIX" era
where new HPC storage models appear in production systems. When consid-
ering what the storage model(s) in this era might look like, two needs are evi-
dent. First, highly parallel applications (and the libraries that support them)
need to store multiple, concurrent streams of data and organize these conve-
niently. Second, with the explosion of data that is occurring, new methods for
identifying data of interest are increasingly important.
30.3.1 Prior Work
Of course, while production HPC systems have primarily provided POSIX
storage access, software products outside of HPC and research in the HPC
