Database Reference
In-Depth Information
Figure 4. Load-balanced chunk processing algorithm
ending, the node proceeds to process a merge query
and then to send the results to a merger node. The
controller keeps track of how many nodes ended
and ends the processing of the chunks as soon
as all nodes ended. As soon as all results are in
the merger node, the controller commands it to
process the merge. In Figure 4 we focus only the
chunk-processing part.
This on-demand load-balanced algorithm to-
gether with a replica placement approach is very
flexible, in the sense that if any node gets delayed
or is unavailable, other nodes holding its replicas
are ready to help it as long as they finished their
own work. This way, the algorithm handles every
kind of heterogeneity and non-dedication delays
gracefully. It also handles availability exceptions:
If a node or a network segment becomes unavail-
able, the communication layer raises an exception
and, after trying to reconnect, the controller flags
it as unavailable. If the node was processing a
query, the chunks processed by it are re-habilitated
and the nodes can immediately start processing
the corresponding chunks for which they may
have a replica. Nodes that had already ended their
work for the query are also waken-up if they have
replica chunks of the failing node.
The controller node can be chosen manually,
and it may either be processing queries as well
or only controlling. In a configuration with a few
nodes, the bulk of the work is expected to be the
processing and merging of queries/chunks, so that
the coordinator functionality is expected to be light
in comparison to that. Still, if there are many nodes,
the controller may have a heavy burden receiving
requests and answering them. We plan to explore
this scalability issue in future work.
CHUNKSIM PERFORMANCE
AVAILABILITY ANALYSIS
In this section we describe the ChunkSim model
and tool. Section 4.1 describes parameters used by
ChunkSim and section 4.2 describes data alloca-
tion and replication alternatives considered by the
model for the SN system. Section 4.3 describes
the experiments that ChunkSim implements to
estimate performance and availability for system
Search WWH ::




Custom Search