Database Reference
In-Depth Information
The MapReduce processing layer (the JobTracker server) also has a single
point of failure.
In Hadoop 1.0 deployments, you can address the NameNode availability
problem through active/passive failover solutions. One option is to automati-
cally replicate data between two separate Hadoop clusters. Another option is
to have a dedicated backup for the master node (which includes the NameNode,
and can also include the JobTracker service), so that all of the Hadoop
cluster's NameNode metadata is backed up. In the event of a NameNode
failure, the Hadoop cluster can restart, using the backup NameNode.
In Hadoop 2.0 (which was in alpha state at the time of writing), there are
two significant availability improvements: the ability to designate a hot
standby for the HDFS NameNode, and YARN (also known as MapReduce2),
which distributes JobTracker functionality so that this server is no longer a
SPOF. IBM committers are working with the open source community to make
Hadoop 2.0 production-ready, and, of course, IBM intends to keep BigInsights
current with the latest enterprise-ready open source innovations.
In the open source community, there is some controversy as some vendors
have released Hadoop 2.0 code in the Generally Available (GA) editions of their
products. The point of contention is that the Hadoop 2.0 code being included
here is not classified as production-ready. Other vendors, including IBM and
Hortonworks, have shied away from this approach, reaffirming their policies
to release only production-ready open source code in their distributions.
We'd love to get into more depth and describe how HDFS and MapRe-
duce work, not to mention the additional projects in the Apache Hadoop
ecosystem, but space is limited. To learn about Hadoop in greater detail, visit
BigDataUniversity.com, where you can find a number of quality online
courses for free!
What's in the Box: The Components of
InfoSphere BigInsights
Before describing the enterprise-readiness capabilities of BigInsights, we'd like
to establish a context, and go over the product's main components from both a
licensing and packaging perspective, and a software component perspective.
We'll describe the Hadoop components included in BigInsights, the main
user interface for BigInsights (the BigInsights Web Console), the developer
tooling, and the various editions that are available.
 
Search WWH ::




Custom Search