Database Reference
In-Depth Information
units belonging to a particular server being stored together on a local disk at
that server. In PVFS the container for the stripe units is called a “datafile,”
and the current implementation stores these datafiles in a directory structure
on a local file system (e.g., XFS). A set of parameters, stored with the meta-
data for a file, defines how the bytes of a particular file are mapped into the
datafiles stored on different servers. Clients access files by requesting reads or
writes of byte regions of these datafiles, and accesses do not need to be aligned
on block or page boundaries.
PVFS servers can be configured to manage data, metadata, or both. At
configuration time the administrator chooses which servers will be responsi-
ble for what types of data. This approach allows the administrator to take
advantage of special hardware, such as solid state disks, or to place data based
on reliability of the underlying hardware.
Because PVFS is freely available, it is often installed on commodity hard-
ware as a scratch file system. In these configurations PVFS is not fault toler-
ant: the loss of a server holding metadata will likely cause the file system to
become unusable until that server is returned to service, and the loss of a disk
is likely to lose data permanently. However, PVFS may also be configured
as a fault-tolerant system when appropriate hardware is available. Using a
RAID volume for underlying storage will allow a PVFS file system to tolerate
disk failures. Using hardware with multiple paths to the underlying storage,
in conjunction with heartbeat software, will allow PVFS to tolerate server
and network link failures as well. (Such a configuration is discussed in greater
detail in the next section.)
One characteristic of PVFS that separates it from most other parallel file
systems is its lack of critical state stored on clients. By eliminating this type
of data on clients, client failures have no significant impact on a running
file system. In contrast, systems like GPFS must correctly detect client fail-
ures and recover from these failures, for example, by reclaiming locks that
the client might have held, in order to ensure that other clients can access
files and directories that were being accessed by a dead client. This ability
to ignore client failures becomes increasingly important as systems are built
with ever larger numbers of clients, because of the increasing chance of client
failures.
Because PVFS does not implement locks (those would be critical state on
a client), coherent caching and buffering on the client are not implemented.
Hence, all client I/O operations must pass over the network, and as a result
small file operations can be slower than on systems such as GPFS where
caching and buffering can hide this latency. For input datasets that will not
change, PVFS does allow the user to mark these files as
immutable
, and PVFS
will cache data read from these files on clients for performance purposes,
providing some of the benefits of caching without the overhead and consistency
complications of locking. Likewise, metadata operations such as creating a new
file are performed atomically, so locks are not needed.
