Hardware Reference
In-Depth Information
of concurrency without manual intervention by the programmer. This feature
makes it very easy for users to modify the file system, even when adding new
requests and developing new optimizations and interfaces, without needing to
understand all of the operational details. Many features in the current code
were developed for this level of usability. For example, large data transfers in-
volve setting up a ow|a fairly complex process. Very small reads and writes
could be performed without a ow|by simply including them in the request
or response. Thus, a new request, SMALLIO, was developed to take advantage
of this optimization. State machines are also important for maintenance and
debugging. In many cases, problems can be found and fixed without debugging
more than a small fraction of the code.
10.2.8 Distributed File Metadata
OrangeFS differs from many file systems in that it can distribute file meta-
data among multiple servers. Metadata for each file will be stored on a single
server (often referred to as \the" metadata server, only with respect to a
given file). The metadata for all files and directories is distributed among
a designated set of servers using a random variable to select the metadata
server each time a file is created. The configuration file allows as few as one
server to be used for metadata, or as many as all of the servers. Metadata
servers can also be data servers, or they can be used to store only metadata.
The best configuration depends on the situation. When there are large num-
bers of users accessing a large number of smaller files, more metadata servers
provide more parallelism. On the other hand, when servers are doing both
data and metadata service, there is the potential for interference. In a situ-
ation where a few users are working with very large files, a single dedicated
metadata server may be better. The cost of implementing the file server will
also affect how they are configured. OrangeFS provides as many options as
possible.
10.2.9 Distributed Directory Entry Metadata
One of the most recent additions released in OrangeFS v2.9 is distributed
directories. In earlier versions, each directory is implemented as a single object
that holds all directory entries on the same server as the directory metadata.
Newer applications involve large numbers of processes that access an extremely
large number of files. In this case, providing parallelism at the directory level
allows more tasks to look up files at the same time. This is implemented by
having multiple objects on different servers hold the directory entries. The
various entries are assigned to one dirdatafile or another using a hash. This is
further optimized by using extensible hashing based on Giga+ [6], which allows
small directories to have just one dirdatafile and then add more dirdatafiles
(and more parallelism) as the number of items in the directory grows.
