Database Reference
In-Depth Information
1.1 Big Data Overview
Data is created constantly, and at an ever-increasing rate. Mobile phones, social
media, imaging technologies to determine a medical diagnosis—all these and more
create new data, and that must be stored somewhere for some purpose. Devices and
sensors automatically generate diagnostic information that needs to be stored and
processed in real time. Merely keeping up with this huge influx of data is difficult,
but substantially more challenging is analyzing vast amounts of it, especially when
it does not conform to traditional notions of data structure, to identify meaningful
patterns and extract useful information. These challenges of the data deluge present
the opportunity to transform business, government, science, and everyday life.
Several industries have led the way in developing their ability to gather and exploit
data:
• Credit card companies monitor every purchase their customers make and
can identify fraudulent purchases with a high degree of accuracy using rules
derived by processing billions of transactions.
• Mobile phone companies analyze subscribers' calling patterns to
determine, for example, whether a caller's frequent contacts are on a rival
network. If that rival network is offering an attractive promotion that might
cause the subscriber to defect, the mobile phone company can proactively
offer the subscriber an incentive to remain in her contract.
• For companies such as LinkedIn and Facebook, data itself is their primary
product. The valuations of these companies are heavily derived from the
data they gather and host, which contains more and more intrinsic value as
the data grows.
Three attributes stand out as defining Big Data characteristics:
•
Huge volume of data:
Rather than thousands or millions of rows, Big
Data can be billions of rows and millions of columns.
•
Complexity of data types and structures:
Big Data reflects the variety
of new data sources, formats, and structures, including digital traces being
left on the web and other digital repositories for subsequent analysis.
•
Speed of new data creation and growth:
Big Data can describe high
velocity data, with rapid data ingestion and near real time analysis.
