Databases Reference
In-Depth Information
Mike Burrows. “The Chubby Lock Service for Loosely-Coupled Distributed Systems”; pub.
OSDI'06: Seventh Symposium on Operating System Design and Implementation, Seattle,
WA, November 2006. URL:
http://labs.google.com/papers/chubby.html
➤
If at this stage or later in this chapter, you are thoroughly confused and over-
whelmed by the introduction of a number of new terms and concepts, hold on
and take a breath. This topic explains all relevant concepts at an easy pace. You
don't have to learn everything right away. Stay with the fl ow and by the time
you read through the topic, you will be able to understand all the important
concepts that pertain to NoSQL and big data.
The release of Google's papers to the public spurred a lot of interest among open-source
developers. The creators of the open-source search engine, Lucene, were the fi rst to develop an
open-source version that replicated some of the features of Google's infrastructure. Subsequently,
the core Lucene developers joined Yahoo, where with the help of a host of other contributors, they
created a parallel universe that mimicked all the pieces of the Google distributed computing stack.
This open-source alternative is Hadoop, its sub-projects, and its related projects. You can fi nd
more information, code, and documentation on Hadoop at
http://adoop.apache.org
.
Without getting into the exact timeline of Hadoop's development, somewhere toward the fi rst of
its releases emerged the idea of NoSQL. The history of who coined the term
NoSQL
and when is
irrelevant, but it's important to note that the emergence of Hadoop laid the groundwork for the
rapid growth of NoSQL. Also, it's important to consider that Google's success helped propel a
healthy adoption of the new-age distributed computing concepts, the Hadoop project, and NoSQL.
A year after the Google papers had catalyzed interest in parallel scalable processing and non-
relational distributed data stores, Amazon decided to share some of its own success story. In
2007, Amazon presented its ideas of a distributed highly available and eventually consistent data
store named Dynamo. You can read more about Amazon Dynamo in a research paper, the details
of which are as follows: Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan
Kakulapati, Avinash Lakshman, Alex Pilchin, Swami Sivasubramanian, Peter Vosshall, and Werner
Vogels, “Dynamo: Amazon's Highly Available Key/value Store,” in the Proceedings of the 21st ACM
Symposium on Operating Systems Principles, Stevenson, WA, October 2007. Werner Vogels, the
Amazon CTO, explained the key ideas behind Amazon Dynamo in a blog post accessible online at
www.allthingsdistributed.com/2007/10/amazons_dynamo.html
.
With endorsement of NoSQL from two leading web giants — Google and Amazon — several
new products emerged in this space. A lot of developers started toying with the idea of using these
methods in their applications and many enterprises, from startups to large corporations, became
amenable to learning more about the technology and possibly using these methods. In less than 5
years, NoSQL and related concepts for managing big data have become widespread and use cases
have emerged from many well-known companies, including Facebook, Netfl ix, Yahoo, EBay, Hulu,
IBM, and many more. Many of these companies have also contributed by open sourcing their
extensions and newer products to the world.
