Database Reference
In-Depth Information
The key to successfully horizontally scaling a system is essentially an
exercise in limiting the amount of coordination required between systems.
This limited coordination applies to both systems communication between
different instances as well as those potentially on the same instances. One of
the chief mechanisms used to accomplish this feat is the use of partitioning
techniques to divide the work between machines. This ensures that a
particular class of work goes to a well-known set of machines without the
need for anyone to ask a machine what it is doing.
When coordination is required, the problem can become fairly complicated.
There have been a number of algorithms developed over the years that
are designed to allow loosely coupled machines to coordinate and handle
events such as network outages. In practice, they have proven devilishly
difficult to implement. Fortunately, these algorithms are now offered “as a
service” through coordination servers. The most popular is the ZooKeeper
system developed by Yahoo! and discussed in great detail in Chapter 3.
Used by many of the other frameworks and software packages in this topic,
ZooKeeper greatly simplifies the task of efficiently coordinating a group of
machines when required.
Languages for Real-Time Programming
There are a number of programming languages in use today, but only a
few of them are widely used when implementing real-time streaming
applications. Part of this is due to the fact that the core packages are
implemented in a specific language, and the focus for the client libraries for
thatpackage is,ofcourse,onitsnative language. Inothercases,thespeedor
ease of use of the language makes it well suited for implementing real-time
applications. This section briefly discusses some of the languages that are
often encountered in real-time applications.
Java
The Java language was made public in the mid-1990s with the promise of
“Write Once Run Anywhere” software. Initially, this was used to embed
Java software into web pages in the form of applets. The plan was to let
the web server act as the distribution mechanism for “fat” client-server
applications. For a variety of reasons, this never really caught on, and Java
is now disabled by default in many web browsers. What functionality it
originally sought to provide was mostly taken over by Adobe's Flash product
