Database Reference
In-Depth Information
The problem is that many of these initiatives have stagnated in the “science
project” phase. The challenges are common: it's easy and exciting to start
dumping data into these repositories; the hard part comes with what to do
next. Meaningful analysis of data that is stored in Hadoop requires highly
specialized programming skills—and for many algorithms, it can be very
challenging to put them into a parallelizable form so that they can run in
Hadoop. And what about information governance concerns, such as security
and data lifecycle management, where new technologies like Hadoop don't
have a complete story?
IBM sees the tremendous potential for technologies such as Hadoop to
have a transformative impact on businesses. This is why IBM has scores of
researchers, developers, and support staff continually building out a platform
for Hadoop, called IBM InfoSphere BigInsights (BigInsights). BigInsights was
released in October, 2010, with a relatively straightforward goal: make
Hadoop enterprise-ready. We describe three main aspects of enterprise readi-
ness in this chapter:
•
Analyticssupport
Enable different classes of analysts to get value
out of data that is stored in BigInsights without asking them to pass
requirements over the fence to Hadoop programmers, or buying
expensive consulting hours to procure custom Hadoop applications
that no one in your organization understands. Think consumability!
•
Dataintegration
If your business is analyzing data that is stored in
Hadoop, you'll need to ensure that your Hadoop system integrates
with the rest of your IT infrastructure. For example, you'll need to be
able to query Hadoop data from your warehouse environment, and,
inversely, query warehouse data from your Hadoop environment.
•
Operationalexcellence
After you start depending on analytics that is
generated by Hadoop, you will need to govern and administer the
data that is stored in Hadoop. BigInsights includes a number of key
features for security, governance, administration, and performance.
Before we describe the BigInsights extensions to Hadoop, you'll need to
understand what Hadoop is in the first place.
