Database Reference
In-Depth Information
In 2005, there were an estimated 1.3 billion RFID tags in circulation; by the
end of 2011, this number had risen to 30 billion! Now consider that RFID
price points are expected to drop below 1 US cent by 2015, and that there are
all kinds of other sensory and measurement technologies already available;
in fact, we'd argue at this point, the world can pretty much measure almost
anything it wants.
From an instrumentation perspective, what doesn't have some amount of
coding in it today? Just look at your car: you can't diagnose a problem these
days without hooking a computer up to it. Today's hardware network switches
have more software code than components in them. The latest passenger air-
planes are instrumented with over one billion lines of code that generates
about 10 terabytes (TB) of data, per engine, during every half hour of opera-
tion. Let's put that in perspective: A single flight from London's Heathrow
Airport to John F. Kennedy in New York would generate about 650TB of data!
That's likely more data than what you have sitting in your warehouse today.
Most of this data is likely never looked at, unless disaster strikes. Imagine the
efficiencies, potential disaster prevention, insights, and other optimization
opportunities if all of the data were cost-effectively analyzed.
One important enterprise differentiator is the ability to capture data that's
getting “dropped to the floor”; this type of data can yield incredible insights
and results, because it enriches the analytics initiatives going on in your
organization today.
Data exhaust
is the term we like to use for this kind of
data: it's generated in huge amounts (often terabytes per day) but typically
isn't tapped for business insight. Online storefronts fail to capture terabytes
of generated clickstreams to perform web sessionization, optimize the “last
mile” shopping experience, and perhaps understand why online shopping
baskets are getting abandoned. Mountains of data could be collected and
analyzed to judge the health of an oil rig. Log files for your most important
networks could be analyzed for trends when nothing obvious has gone
wrong to find the “needle in the stack of needles” that potentially indicates
downstream problems. There's an “
if
” here that tightly correlates with the
promise of Big Data: “
If
you could collect and analyze all the data…” We like
to refer to the capability to analyze all the data as
whole population analytics
.
It's one of the value propositions of Big Data. It's a consideration on the kind
of predictions and insights your analytic programs could make if they
weren't restricted to samples and subsets of the data.
