Database Reference
In-Depth Information
Hadoop engine is to provide database archiving services to a data warehouse,
taking data that's no longer “warm” or “hot” and moving it to a lower-cost
storage platform that is backed by Hadoop. For example, a customer profiling
system keeps hot data for two years, but all of the data collected over the
course of a 20-year relationship might still be valuable. An insurance company
could benefit from understanding how your profile has evolved from being a
single person, then married, then with kids, while considering current trends
or events (this is especially true with wealth management portfolios).
Of course, the portability of the query API is of key importance in this
scenario; for example, not having to recode your SQL-based applications
when accessing cold data that was moved to Hadoop (the IBM Big Data plat-
form lets you do this). Another example, and Big Data platform requirement,
is the integration between the SQL and NoSQL worlds. You might have a
consumer vulnerability job running in your relational warehouse but choose
to launch a Hadoop-based sentiment job on your brand that could impact the
final vulnerability assessment. At the same time, you might have a running
Hadoop job that is analyzing terabytes of clickstream log data, and you now
want to extract purchase information from the system of record to under-
stand what other factors lead to a successful purchase or abandonment of
past online shopping carts. (Now consider if you can take that logic and
apply it to shopping carts as they are being populated in real time.)
Traditional engines and new data processing engines (Hadoop and others)
will become the left and right arm of an organization. The key is ensuring that
your Big Data platform provider delivers integration technologies that enable
those arms to be used in tandem.
As a final analogy, consider a baseball player. A typical baseball player is
very strong at throwing with one hand and catching with the other; the brain
coordinates the activities of these limbs for optimized results. If a baseball
player were to try to throw or catch a ball with his nondominant hand, he
might be able to do it, but it won't be smooth, crisp, or look very professional.
And with the exception of a few professional baseball players who have only
one hand (such as Jim Abbott,
www.jimabbott.net)
, you don't really see baseball
players catch a ball, and then take their glove off to throw the ball with the same
hand. This is the NoSQL and SQL analogy; each is optimized for specific tasks
and data and it's important to play ball with two hands.
