Hardware Reference
In-Depth Information
do not justify the effects. One of the easiest ways to render a file unreadable
and crash a process is writing to a nearly full disk. In the case of stochastic
simulations, it is often impossible to predict the exact amount of disk space
that the simulation will need. In case there is not enough space, customers
loose days of work and CPU cycles.
20.4.5 Fragmentation
In the cases where the files are no created for warehousing, the same file will
be used to run simulations multiple times. This results in data being deleted
and rewritten. HDF5 does not provide the means to eciently recover the
free space in the file, often leading to excessive fragmentation and increased
TCDO.
20.4.6 Process and Thread Synchronization
HDF5 does not provide any means to synchronize processes. Processes are
free to access a file and the results are undefined. Furthermore, opening a
file from synchronized processes using file locking on open handles does not
ensure that after a process has written additional data to the file, the file
view of other processes can still access all the data before the write. Thread
synchronization is an issue with HDF5 and it is not trivial to ensure that it
always works. Further, there are cases where static symbol in the library cause
anything from memory leaks to crashes.
20.5 A (Near) Ecient Architecture Using HDF5
Considering all the pros and cons of the HDF5 library, it is a relatively good
platform to implement a persistence system for parallel computing, within the
described context. It does, nevertheless, need a lot of additional development,
exploitation of open ends, and extensive stress and starvation testing, to en-
sure the quality levels that industrial users require. Figure 20.1 shows an
architecture that describes how HDF5 can be used to allow ecient access to
files from multiple processes and threads in a parallel computing environment
of stochastic FE simulations. The intimidating nature of this figure is indica-
tive of the amount of work that is needed to get HDF5 to deploy its greatness
as a tool for large dataset I/O.
At the lowest level, before accessing OS I/O functions, there is an obscure
component designated \detours." This is a component that exploits the nature
of VFL drivers, which is that the driver itself is a set of function pointers. A
malicious user of the library could detour these functions to his own functions
 
Search WWH ::




Custom Search