Information Technology Reference
In-Depth Information
front-end
back-end
Data Handler
Data Service
«use»
Document
Variable
Task
Image
File
Fig. 2.
Shared storage system schema
a.
Front - end
.A
Document Handler
is defined to provide a general API for
document operations. It acts as a proxy which allows to handle different data
types seamlessly and independently of the back-end implementation.
b.
Back-end
.Itiscomposedofa
Document Service
and runs in background
on a defined node or can be splitted using sharding when available, allowing
splitting the data volume and computational load over multiple servers.
3 Implementation Topics
The strategy presented so far is quite straightforward because it is based on
standard design patterns [10]. However, it is worth to mention four critical im-
plementation aspects. Deployment details are shown in Figure 3.
a.
Messaging system.
A JSON based message specification that includes both
internal and external events was defined. This satisfies the requirement to have
heterogeneous interoperability. A simple protocol was defined which, despite
it is lightweight, it is flexible enough to contain a rich message set — ranging
from low level messages (system calls, synchronization messages or callbacks)
to user defined commands. ZeroMQ [8,12] was chosen for message transport,
that implements IPC socket for internal messaging and TCP sockets for node
communications. It also provides fault tolerance functionalities, to deal with
issues like
slow joined
or sockets disconnections.
b.
Parallel and distributed processing.
Each node contributes with a num-
ber of Workers (running as separated processes) and an implementation of a
Task Service (running as background server). They are all synchronized using
message passing through the messaging system, avoiding classic approaches
with low level structures such as semaphores or mutex [13]. Every node can
