Standardized Multi-protocol Data Management for Grid and Cloud GridRPC Frameworks - Data Management in Cloud, Grid and P2P Systems - page 67

computation for example, there is a need to keep data in memory and avoid
file transfers. The GridRPC Data Management API foresaw this kind of use
and introduced the memory protocol. In addition to this protocol management,
our implementation lets a client (or the library itself) use URIs with query and
fragment. This leads to possible evolution for improvements (see after) and to
manage more data managers (like P2P middleware that initiate torrents with
specific files).
Implementing the memory protocol means that the library has to use GridRPC
middleware inner data manager which can hopefully communicate between its
own components to achieve such a need. But when a data is in memory and has
to be transferred either on another GridRPC middleware components or on a
storage server, it has to be written to a file and then be manipulated (transferred
and possibly handled remotely) to be in the requested status. This part uses
(de)serialization functions, defined by the GridRPC Data Management API,
partly relying on tools provided by the boost library. But that maybe shows an
unclear part of the API: the protocol to use in that specific case to manipulate
the file is not precised. In our current implementation, the protocol is static and
is read from the middleware configuration file at initialization time.
But if going a bit further than the API, we can use the query part of the
URI. Indeed considering a data available in memory, the API does not provide
a mean to know which protocol(s) can be used to send or to receive it since the
URI would be similar to memory://graal.ens-lyon.fr/matrixA . In ongoing
work, our library is going to explore what can be done with specifying protocols
within the URI query part, e.g., ?protocol=rsync?protocol=webdav .
Scheduling for Implicit and Explicit Data Transfers. Data transfers are
operated 1) when data participate to a remote procedure call. They are in that
case implicit or automatic, and; 2) can be explicitly requested by a client with
acallto grpc_data_transfer() .
Implicit and Automatic Transfers: When a remote procedure call is performed,
meta-data are serialized and transferred to the distant service. They contain
sets of URIs which may lead to additional transfers before and after the service
execution (Fig 2).
- If one of the input URI refers to a memory or file data on the client, the data
must be available to the service before its execution so that it can remotely
access it.
- If one of the ouput URI refers to a memory or file data on the client, the
service must have made it available and the client must get it.
Explicit Data Transfers: Several transfers are operated by
grpc data transfer() , i.e., a call to an explicit transfer operation: the
data should be present in all locations set in the input URIs list, and must
be present in all locations set in the output URIs list. This can be treated
with a sequential set of transfers from one given source to each destination for