Database Reference
In-Depth Information
Small I/O requests were common; although programmers can aggregate re-
quests in the application, they often did not.
From conversations with the application programmers, the group found
that the I/O capabilities of the system did not match the desired ones. This
mismatch resulted in complications in application code and also reduced the
scope of feasible problems. For example, in two of the applications the pro-
grammers found it easier to read data into a single node and then distribute
to the remaining nodes rather than use parallel access modes, because the
available modes on the Paragon did not allow for the particular pattern of
access they needed.
Since these studies, many advances have been made in I/O middleware and
high-level I/O libraries to help applications more effectively access storage.
In particular, efforts have been made to make it easier to describe strided ac-
cess patterns and the relationships between the accesses of different processes.
However, we still see applications that write a file from each process and other
applications that perform all I/O through one process, and neither of these
I/O approaches scales well. As these applications attempt to execute at larger
scale, we hope that better I/O tools will be adopted.
2.4.2 POSIX I/O
The standard interface for file I/O is defined as part of the POSIX standard.
11
The interface was developed at a time when file systems were typically local
to a single operating system instance, and concurrent access occurred only
between multiple processes running on the same computer.
The interface provides a simple set of functions for accessing contiguous
regions of files as well as functions for creating and removing files and direc-
tories and traversing the contents of directories. It further defines a strict set
of semantics for these operations, guaranteeing sequential consistency of I/O
operations in all cases, meaning that all operations appear to have occurred
one at a time in some sequential ordering that is consistent with the ordering
of the individual operations. So, if two processes simultaneously write to the
same region, it will appear as if one wrote first, and then the other second.
Likewise, if one process reads while the other writes, the reader will see either
all or none of the writer's changes. This API provides a convenient mechanism
for serial applications to perform a variety of common tasks, and in the UNIX
environment it is also the mechanism through which many system activities
are performed (by treating devices as special types of files). The limitations
of the POSIX interface become apparent only when a file system is made
available over a network or simultaneously made available to more than one
operating system instance.
When an operating system is interacting with a file system that is attached
locally, the latency of communication with that file system is very short. In
this environment, the simple building blocks that POSIX provides for per-
forming I/O are more than adequate, because the cost of assembling more
