Database Reference
In-Depth Information
request to just one instance. On the contrary, each request is propagated in a
round-robin manner to only one instance belonging to the load balanced group.
Moreover, through experiments, we have also noticed that co-locating LMS with
Virtuoso created memory issues in cases of demanding queries and increased
query load as the container in which LMS was deployed required at least 2 GB
of main memory. To solve this problem, it was decided to use LMS as a first
entrance of user requests which will process them and then decide to connect:
(a) either to the Virtuoso residing at the master instance in case of update
requests or (b) to the Amazon Load Balancer which will redirect the connection
request to the next instance in the load balancing group in case of query or
export requests. This solution has the advantage that LMS can base its decision
upon the respective LMS method called and thus it is easy to implement the
connection policy to the appropriate Virtuoso instance. To not consume many
resources which are not actively exploited (i.e., in the case of infrequent updates
to the master instance), it was decided to include the master instance in the load
balancing group of Amazon LB which of course also includes one or more slave
instances that are inserted or removed from it in terms of particular scale-in
and scale-out policies/rules. As such, query/export requests are also answered
by the master instance. The LB group was set to have at least one instance and
to remove the newest instances developed when they are no more needed so that
the master instance is always present even when one instance has been left in
the group.
Based on the above analysis, the following sub-section provides an insight on
the new architecture proposed and its main advantages.
5.2 Architecture
Figure 2 shows the new architecture of our LD Management System. As it can be
seen, the topmost component is LMS which is regarded as a first level of user-
request processing and which issues connection requests to either the master
instance or to Amazon LB depending on the type of method requested. Ama-
zon LB is regarded as the second level of user-request processing and routes
each incoming connection to that Virtuoso engine running in the appropriate
instance from its group. Then the respective instance selected constitutes the
final processing level for user requests where low level RDF management com-
mands are executed in the underlying Virtuoso server. In case of updates, only
the master instance is selected and after the update of respective Virtuoso server,
the triggers produce the respective entries in the log file which are consumed
by the Updater in order to update the Virtuoso servers of the slave instances.
The Updater also updates the slave image every half an hour only when updates
have occurred in the meantime between the latest image update.
To further enhance the availability of the system, each instance (slave or mas-
ter) includes a small script which checks the availability of the respective com-
ponent implementations included in it: (a) the Virtuoso server and (b) Updater
in case of a master instance, and reboots them if needed. In addition, the master
instance is backed up via LMS by a back-up instance which temporarily takes
Search WWH ::




Custom Search