Databases Reference
In-Depth Information
Chapter 1
“Big Data” in the Enterprise
Humans have been generating data for thousands of years. More recently we have seen
an amazing progression in the amount of data produced from the advent of mainframes
to client server to ERP and now everything digital. For years the overwhelming amount
of data produced was deemed useless. But data has always been an integral part of every
enterprise, big or small. As the importance and value of data to an enterprise became
evident, so did the proliferation of data silos within an enterprise. This data was primarily
of structured type, standardized and heavily governed (either through enterprise wide
programs or through business functions or IT), the typical volumes of data were in the
range of few terabytes and in some cases due to compliance and regulation requirements
the volumes expectedly went up several notches higher.
Big data is a combination of transactional data and interactive data. While
technologies have mastered the art of managing volumes of transaction data, it is the
interactive data that is adding variety and velocity characteristics to the ever-growing data
reservoir and subsequently poses significant challenges to enterprises.
Irrespective of how data is managed within an enterprise, if it is leveraged properly,
it can deliver immense business values. Figure
1-1
illustrates the value cycle of data,
from raw data to decision making. In the early 2000s, the acceptance of concepts like
Enterprise Data Warehouse (EDW), Business Intelligence (BI) and analytics, helped
enterprises to transform raw data collections into actionable wisdom. Analytics
applications such as customer analytics, financial analytics, risk analytics, product
analytics, health-care analytics became an integral part of the business applications
architecture of any enterprise. But all of these applications were dealing with only one
type of data: structured data.
