Hardware Reference
In-Depth Information
Thread synchronization ensures that each process accesses a file through
the same file driver and all threads of the process share that file driver. The file
driver concept also implements thread access policies. Each file driver allows
write access to one thread at a time, but may allow read access to multiple
threads.
Finally, at the top level of the architecture, transactions are defined and
managed. These, in principle, use the Data Access Object (DAO) pattern.
A DAO implements a set of transactions through its methods. Each trans-
action needs to lock a file context, perform the transaction, and release the
file context. This ensures that thread and process synchronization policies
from lower levels are respected. At this level, there is also a facility to bundle
DAO transactions together in a single composite transaction. This ensures
that consecutive transactions that assume data presence reported by previous
transactions do not get surprised if another process deleted that data. It also
ensures that the overhead of opening, locking, and closing files, along with
refreshing HDF5 structures, is needed only once.
20.6 Performance
Despite the complexity of the proposed architecture, practical application
shows good performance in terms of file access, especially over networks, and
increased robustness of the HDF5-based persistence implementation. Perfor-
mance in this context is measured in terms of user perceived performance,
user experience, comparison to other applications, and user expectations from
a particular action, so measurements are quite dependent on the application
and its interaction with the user.
The absence of figures and numbers in this section underpins that it is
particularly dicult to offer quantified results regarding performance in such
use cases. The reason is that the factors that determine performance are nu-
merous and adverse. Internally, the system attempts to write from different
processes data generated in or required from different threads in different ac-
cess patterns. While simulation processes or proxies write simulation data in
the file, other processes read this data across multiple simulations to perform
their own tasks while threads of these processes probe for statuses, write or
read logging information, etc. The system needs to respond eciently to all
kinds of transactions of different sizes. Externally, the system is installed on
disparate logical and physical environments. Customers may use any kind of
configuration to run a stochastic simulation: files can be local or on a file
server; network systems can be of any kind with often detrimental effects in
process coordination; different mixes of machines need to access the file; dif-
ferent job schedulers such as LSF, Condor, and others are used to submit
jobs; network latency is not predictable; all kinds of processes may interfere,
 
Search WWH ::




Custom Search