Database Reference
In-Depth Information
If you don't define lifecycle policies and enforce them with automated tech-
nology, you'll either become overwhelmed with Big Data accumulation, or
your administrators will expend a lot of manual effort determining how and
when to delete or retire this data. Consider social media data—how long do
you need to keep it? What about the insights derived from it—how long are
they relevant? At the same time, some of the Big Data promise is to keep a
corpus (complete history) of information, so while you may never delete this
data, it still has a temperature associated with it.
Many use cases for Big Data involve analyzing sensitive information. Orga-
nizations must define security policies to safeguard such information, and
those policies must be monitored and enforced.
The integrity aspect of Big Data is such a hot topic that it ended up with a
sexy term,
veracity
, which we introduced in Chapter 1. You need to determine
whether your Big Data should be cleansed with the same approach that you
would apply to your traditional data, or whether you risk losing potentially
valuable insights by cleansing it. The answer depends entirely on what
you're planning to do with this data. Some use cases, such as customer anal-
ysis, require or would at least benefit from higher-quality data. Other use
cases, such as fraudulent identity analysis, which might require analyzing
data exactly as entered to discover false identity patterns, would not depend
on higher-quality data.
Many Big Data use cases center on key master data management (MDM)
concepts, such as customers, products, locations, and suppliers. But many
organizations haven't established a single version of the truth for those
domains before the onset of the Big Data craze. Consider a social media-
based customer profiling application. One of its key starting points is know-
ing your customers. The linkage is that many MDM projects have the goal of
providing a single view of your customers. The linkage between MDM and
Big Data centers on the most valuable business entities that an organization
concerns itself with, including: customers, products, and households. That's
the linkage between MDM and Big Data. MDM is a good starting point for
many Big Data use cases, and it also provides a logical hub to store insights
gleaned from Big Data analytics. For example, if you consider a master data
project that centers around a person, then extracting life events over Twitter
or Facebook, such as a change in relationship status, a birth announcement,
and so on, enriches that master information and acts as a kind of feeder to
