Databases Reference
In-Depth Information
Decision Making
Actionable
Insight
Synthesizing
Knowledge
Analyzing
Summarizing
Information
Organizing
Data
Collecting
Figure 1-1.
Transforming raw data into action-guiding wisdom
The ubiquity of the Internet has dramatically changed the way enterprises function.
Essentially most every business became a “digital” business. The result was a data explosion.
New application paradigms such as web 2.0, social media applications, cloud computing,
and software-as-a-service applications further contributed to the data explosion. These new
application paradigms added several new dimensions to the very definition of data. Data
sources for an enterprise were no longer confined to data stores within the corporate firewalls
but also to what is available outside the firewalls. Companies such as LinkedIn, Facebook,
Twitter, and Netflix took advantage of these newer data sources to launch innovative product
offerings to millions of end users; a new business paradigm of “consumerism” was born.
Data regardless of type, location, and source increasingly has become a core business
asset for an enterprise and is now categorized as belonging to two camps:
internal data
(enterprise application data) and
external data
(e.g., web data). With that, a new term has
emerged:
big data
. So, what is the definition of this all-encompassing arena called “big data”?
To start with, the definition of big data veers into 3Vs (exploding data volumes, data
getting generated at high velocity and data now offering more variety); however, if you
scan the Internet for a definition of big data, you will find many more interpretations.
There are also other interesting observations around big data: it is not only the 3Vs
that need to be considered, rather when the scale of data poses real challenges to the
traditional data management principles, it can then be considered a big data problem.
The heterogeneous nature of big data across multiple platforms and business functions
makes it difficult to be managed by following the traditional data management principles,
and there is no single platform or solution that has answers to all the questions related to
big data. On the other hand, there is still a vast trove of data within the enterprise firewalls
that is unused (or underused) because it has historically been too voluminous and/or raw
(i.e., minimally structured) to be exploited by conventional information systems, or too
costly or complex to integrate and exploit.
Big data is more a concept than a precise term. Some categorize big data as a volume
issue, only to petabyte-scale data collections (> one million GB); some associate big data
