Database Reference
In-Depth Information
The mysql driver library mysql-connector-java-5.1.22-bin.jar in the directory /usr/lib/sqoop/lib that is installed
for Sqoop on hc1nn is copied to the Sqoop2 server hc1r1m1, as follows:
[root@hc1nn lib]# pwd
/usr/lib/sqoop/lib
[root@hc1nn lib]# ls -l mysql-connector-java-5.1.22-bin.jar
-rw-r--r--. 1 root root 832960 Jul 17 18:50 mysql-connector-java-5.1.22-bin.jar
It is copied via ftp to the following directory on hc1r1m1:
/usr/lib/sqoop2/webapps/sqoop/WEB-INF/lib
This MySQL driver gives Sqoop2 the ability to access the MySQL databases from the server hc1r1m1. Having
made these changes, I restart the Sqoop2 server using the Linux server restart command as root:
[root@hc1r1m1 ~]# service sqoop2-server restart
I then update the section in the Hue configuration file hue.ini for Sqoop2 to reflect these changes. The port
number comes from the value of the SQOOP_HTTP_PORT variable in the setenv.sh under /etc/sqoop2/conf on the
sqoop2 install server hc1r1m1:
# Sqoop server URL
server_url=http://hc1r1m1:12000/sqoop
HBase Cluster Setup for Hue
My section for HBase in the hue.ini file has the following entry (which will be used when the HBase cluster is set up).
I determined the HBase port number by looking for the HBase Thrift server port number in the HBase logs. I looked
in the directory /var/log/hbase and searched the log files there for the term “TBoundedThreadPoolServer,” and the
related log message, then provide the port number. The term “Cluster” represents the fact that this is a clustered
version of HBase running on many servers:
hbase_clusters=(Cluster|hc1r1m1:9090)
There are some new HBase servers to introduce in this section, so before I start installing them, let's review their
purposes. The HBase Region server manages the HBase regions comprising the storages files and blocks. The HBase
Thrift server provides a thrift API for HBase; it means that HBase clients can be developed in multiple languages and can
be used to access HBase. The Hbase Rest server uses an HTTP-based method to access HBase, and access is achieved
and data passed via a web address. The HBase Master process is the main server that manages the other servers.
Before you can use the HBase browser in Hue, you must set up HBase to run as a cluster because Hue attempts
to connect to the HBase Thrift server. For this example, I will install and run HBase on the three nodes where my
ZooKeeper servers are running (hc1r1m1, hc1r1m2, hc1r1m3 ). HBase needs a master node in the cluster, so the first
step for me is to install the HBase Master server (hc1r1m1, for the example):
yum install hbase-master
Next, I install the HBase Thrift server on the HBase master node (hc1r1m1) and I install the Rest and Region
servers on all HBase cluster nodes:
yum install hbase-thrift
yum install hbase-rest
yum install hbase-regionserver
 
Search WWH ::




Custom Search