An Open-Source Cloud Architecture for Big Stream IoT Applications - Interoperability and Open-Source Solutions for the Internet of Things - page 77

- a normalization process handles the input data currently available on the
queue and performs all necessary normalization operations in order to obtain
a stream of information units that can be processed by next modules;
- the normalized stream is forwarded to an output exchange;
The output of the Normalization block represents the entry-point of the first Graph
Module Exchange , that pass it to all the interested listeners of the next levels. The
main advantage of using Exchanges is that queues and normalization processes
can be dynamically adapted to the current workload; for instance, normalization
queues and processes could be easily replicated to avoid system congestion.
2.3 Graph Framework
The Graph Framework is composed of listeners. A listener is an entity (e.g.,
a processing unit in the graph or an external consumer) interested in the raw
data stream or in the output provided by a different node in the graph [ 7 ]. Each
listener represents a node in the topology. The connection of multiple listeners
across all processing units define the routing of data streams from producers to
consumers. The nodes are processing units performing some kind of computation
on incoming data and edges represent the flow of information linking together
processing units, which implement some complex behavior as a whole. All the
nodes of the graph can be listeners for incoming data or outputs of other graph
nodes. As shown in Fig. 2 , the graph is divided in two stages: (i) in the Core
Graph basic processing provided by the architecture (e.g., format translation,
normalization, aggregation, data correlation, and other transformations) is per-
formed; (ii) in the Application Graph , listeners that require data coming from the
core or an inner graph level to perform custom processing on already processed
data are defined. The Core and Application graph are organized in one or more
concentric levels, in order to provide a set of commonly available functionalities
and to dynamically extend the capabilities of the system. The complexity of pro-
cessing is directly proportional to the number of levels that data have crossed
and the flow of information is always directed to outer or external levels.
In the “Graph Framework”, each level is accessible from a level-dedicated
Exchange that forwards all data streams to nodes in its level. Each graph node
i in a specific layer n can listen for incoming data stream on a dedicated queue
managed by the Exchange of level n . If the node i , as well as being a consumer,
it acts also as a publisher, then its computation results are delivered to the
Exchange of level n , which is bounded with the Exchange of layer n+1 . Therefore
the Exchange in level n+1 can forward streams coming from level n to all nodes
of level n+1 interested in this kind of data.
2.4 Application Register Module
The Application Register Module has the fundamental responsibility to maintain
all the information about the current state of all graph nodes in the system, and
to route data across the graph. In more detail, the application register module