Persistence and Communication State Transfer in an Asynchronous Pipe Mechanism - Cloud, Grid and High Performance Computing: Emerging Applications - page 247

DESIGN AND IMPLEMENTATION asynchronously. During a pi_attach(), the reader

informs the server of its identity. This allows data
streams to be forwarded to the reader while it is
busy performing computations. A single event-
driven in-bound thread stores incoming segments
into a buffer (by the data store component). When
buffer capacity is reached, data segments are
stored into a local file, identified by the pipe_id.
During a pi_read(), this buffer/file is accessed to
retrieve the requested segment.

The π-Spaces/π-channels model is implemented
as an API with a runtime system, and a multi-
threaded server that provides the dynamic lookup
and storage of π-channels. A π-Server manages
a single π-Space, which encourages deploying
multiple servers to improve the distribution of
load. Furthermore, a π-Server may execute at the
cluster head node, serving as a communication
gateway to processes on remote clusters.

General Description of the
Distributed Algorithm Basic Components and General

Functional Description

Table 2 presents the distributed algorithm for
the π-Spaces/π-channels. We adopt the Python
convention to indicate block structure (i.e., the
statement alignment determines a block). The
notation (#line_no) is used in-text when referring
to the algorithm.
The pair (pipe_id, space_id) represents a
system-wide π-channel identifier, and pipe_id is
unique within each π-Space. Since the algorithm
assumes a single π-Space, we remove reference
to the space_id.
Each participating application maintains the
following variables:

Figure 5 (right) presents the π-Server components.
The server maintains a thread pool for incoming
and outgoing transmissions, providing a non-
archival pipe storage service. The thread count
is configurable to support different application
loads. The in-bound threads handle incoming data
streams from sources, while out-bound threads
forward/push π-channel data to sinks. Data streams
are transported using a TCP protocol, while look-
ups are implemented using a lightweight UDP
protocol with retransmission capability.
Figure 5 (left) shows the client-side compo-
nents. Each π-Spaces/π-channels application is
capable of accepting incoming transmissions

Figure 5. Client-side components and design of the π-server