Databases Reference
In-Depth Information
The use of MapReduce inspired engineers from Yahoo! and other organizations to
create open source versions of Google's MapReduce. It fostered a growing awareness
of the limitations of traditional procedural programming and encouraged others to
use functional programming systems.
1.3.3
Case study: Google's Bigtable—a table with a billion rows
and a million columns
Google also influenced many software developers when they announced their Big-
table system white paper titled
A Distributed Storage System for Structured Data
. The moti-
vation behind Bigtable was the need to store results from the web crawlers that extract
HTML
pages, images, sounds, videos, and other media from the internet. The result-
ing dataset was so large that it couldn't fit into a single relational database, so Google
built their own storage system. Their fundamental goal was to build a system that
would easily scale as their data increased without forcing them to purchase expensive
hardware. The solution was neither a full relational database nor a filesystem, but
what they called a “distributed storage system” that worked with structured data.
By all accounts, the Bigtable project was extremely successful. It gave Google
developers a single tabular view of the data by creating one large table that stored all
the data they needed. In addition, they created a system that allowed the hardware to
be located in any data center, anywhere in the world, and created an environment
where developers didn't need to worry about the physical location of the data they
manipulated.
1.3.4
Case study: Amazon's Dynamo—accept an order 24 hours a day,
7 days a week
Google's work focused on ways to make distributed batch processing and reporting
easier, but wasn't intended to support the need for highly scalable web storefronts that
ran 24/7. This development came from Amazon. Amazon published another signifi-
cant NoSQL paper:
Amazon's 2007 Dynamo: A Highly Available Key-Value Store
. The busi-
ness motivation behind Dynamo was Amazon's need to create a highly reliable web
storefront that supported transactions from around the world 24 hours a day, 7 days a
week, without interruption.
Traditional brick-and-mortar retailers that operate in a few locations have the lux-
ury of having their cash registers and point-of-sale equipment operating only during
business hours. When not open for business, they run daily reports, and perform back-
ups and software upgrades. The Amazon model is different. Not only are their custom-
ers from all corners of the world, but they shop at all hours of the day, every day. Any
downtime in the purchasing cycle could result in the loss of millions of dollars. Ama-
zon's systems need to be iron-clad reliable and scalable without a loss in service.
In its initial offerings, Amazon used a relational database to support its shopping
cart and checkout system. They had unlimited licenses for
RDBMS
software and a
consulting budget that allowed them to attract the best and brightest consultants for
