Database Reference
In-Depth Information
Hadoop Components Included
in InfoSphere BigInsights 2.0
BigInsights features Apache Hadoop and its related open source projects as a
core component. This is informally known as the IBM Distribution for Hadoop.
IBM remains committed to the integrity of these open source projects, and will
ensure 100 percent compatibility with them. This fidelity to open source pro-
vides a number of benefits. For people who have developed code against other
100 percent open source-compatible distributions, their applications will also
run on BigInsights, and vice versa. This open source compatibility has enabled
IBM to amass over 100 partners, including dozens of software vendors, for
BigInsights. Simply put, if the software vendor uses the libraries and interfaces
for open source Hadoop, they'll work with BigInsights as well.
IBM also releases regular product updates for BigInsights so that it remains
current with the latest releases of the open source components.
The following table lists the open source projects (and their versions) that
are included in BigInsights 2.0, which was the most current version available
at the time of writing.
Component
Version
Hadoop (common utilities, HDFS, and the MapReduce framework)
1.0.3
Avro (data serialization)
1.6.3
Chukwa (monitoring large clustered systems)
0.5.0
Flume (data collection and aggregation)
0.9.4
HBase (real-time read and write database)
0.94.0
HCatalog (table and storage management)
0.4.0
Hive (data summarization and querying)
0.9.0
Lucene (text search)
3.3.0
Oozie (work low and job orchestration)
3.2.0
Pig (programming and query language)
0.10.1
Sqoop (data transfer between Hadoop and databases)
1.4.1
ZooKeeper (process coordination)
3.4.3
With each release of BigInsights, updates to both the open source compo-
nents and IBM components go through a series of testing cycles to ensure
that they work together. That's another special point that we want to clarify:
You can't just drop new code into production. In our experience, backward-
compatibility issues are always present in open source projects. BigInsights
pretty much takes away all of the risk and guesswork that's associated with
