Database Reference
In-Depth Information
This is an introduction to the Hadoop V1 and V2 architectures. You might have the opportunity to work with both
versions, so I give examples for installation and use of both. The architectures are obviously different, as seen in
Figures 2-1 and 2-2 , and so the actual installation/build and usage differ as well. For example, for V1 you will carry out
a manual install of the software while for V2 you will use the Cloudera software stack, which is described next.
The Hadoop Stack
Before we get started with the Hadoop V1 and V2 installations, it is worth discussing the work of companies like
Cloudera and Hortonworks. They have built stacks of Hadoop-related tools that have been tested for interoperability.
Although I describe how to carry out a manual installation of software components for V1, I show how to use one of
the software stacks for the V2 install.
When you're trying to use multiple Hadoop platform tools together in a single stack, it is important to know what
versions will work together without error. If, for instance, you are using ten tools, then the task of tracking compatible
version numbers quickly becomes complex. Luckily there are a number of Hadoop stacks available. Suppliers can
provide a single tested package that you can download. Two of the major companies in this field are Cloudera and
Hortonworks. Apache Bigtop, a testing suite that I will demonstrate in Chapter 8, is also used as the base for the
Cloudera Hadoop stack.
Table 2-1 shows the current stacks from these companies, listing components and versions of tools that are
compatible at the time of this writing.
Table 2-1. Hadoop Stack Tool Version Details
Cloudera CDH 4.6.0
Hortonworks Data Platform 2.0
Ambari
1.4.4
DataFu
0.0.4
Flume
1.4.0
1.4.0
Hadoop
2.0.0
2.2.0
HCatalog
0.5.0
0.12.0
HBase
0.94
0.96.1
Hive
0.10.0
0.12.0
Hue
2.5.0
2.3.0
Mahout
0.7
0.8.0
Oozie
3.3.2
4.0.0
Parquet
1.2.5
Pig
0.11
0.12.0
Sentry
1.1.0
Sqoop
1.4.3
1.4.4
Sqoop2
1.99.2
Whirr
0.8.2
ZooKeeper
3.4.5
3.4.5
 
 
Search WWH ::




Custom Search