Information Technology Reference
In-Depth Information
Chapter 15
Persistence and Communication
State Transfer in an
Asynchronous Pipe Mechanism
Philip Chan
Monash University, Australia
David Abramson
Monash University, Australia
ABSTRACT
Wide-area distributed systems offer new opportunities for executing large-scale scientific applications.
On these systems, communication mechanisms have to deal with dynamic resource availability and the
potential for resource and network failures. Connectivity losses can affect the execution of workflow
applications, which require reliable data transport between components. We present the design and
implementation of π-channels, an asynchronous and fault-tolerant pipe mechanism suitable for coupling
workflow components. Fault-tolerant communication is made possible by persistence, through adaptive
caching of pipe segments while providing direct data streaming. We present the distributed algorithm
for implementing: (a) caching of pipe data segments; (b) asynchronous read operation; and (c) com-
munication state transfer to handle dynamic process joins and leaves.
INTRODUCTION
scientific instruments, data repositories, clusters
and supercomputers such as computational grids
(Foster and Kesselman, 1999), these systems are
heterogeneous and dynamic in availability. Fur-
thermore, the wide-area links that interconnect
these resources are prone to transient or permanent
failures. These dynamic characteristics introduce
Heterogeneous distributed systems are the
emergent infrastructures for scientific comput-
ing. From peer-to-peer, volunteer computing
systems to the more structured ensembles of
Search WWH ::




Custom Search