Database Reference
In-Depth Information
Nutch and Solr Configuration
You have already learned how to install Nutch and Solr in the first architecture example, so I will be brief here. To
begin, you set up a little configuration in the Linux hadoop account's $HOME/.bashrc for Solr and Nutch.
The following Bash file defines the Hadoop and Java variables; it also creates some useful aliases for Hadoop
commands, like fs (saves typing). In addition, it sets up the shell search path ( PATH ) and defines some variables for
Solr and Nutch.
[hadoop@hc1r1m2 ~]$ cd $HOME
[hadoop@hc1r1m2 ~]$ ls -l .bashrc*
-rw-r--r--. 1 hadoop hadoop 1142 Apr 5 15:57 .bashrc
[hadoop@hc1r1m2 ~]$ cat .bashrc
#######################################################
# Set Hadoop related env variables
export HADOOP_PREFIX=/usr/local/hadoop
# set JAVA_HOME (you will also set a hadoop specific value later)
export JAVA_HOME=/usr/lib/jvm/jre-1.6.0-openjdk
# some handy aliases and functions
unalias fs 2>/dev/null
alias fs="hadoop fs"
unalias hls 2>/dev/null
alias hls="fs -l"
unalias cdh 2>/dev/null
alias cdh="cd $HADOOP_PREFIX"
# add hadoop to the path
export PATH=$HADOOP_PREFIX:$PATH
export PATH=$HADOOP_PREFIX/bin:$PATH
export PATH=$HADOOP_PREFIX/sbin:$PATH
#######################################################
# Set up Nutch and Solr variables
export NUTCH_HOME=/usr/local/nutch
export NUTCH_CONF_DIR=$NUTCH_HOME/conf
export SOLR_HOME=/usr/local/solr
export PATH=$PATH:$NUTCH_HOME/bin
You now install Nutch 2.x from Apache on the machine hc1r1m2, because that is where ZooKeeper is already
installed. For example, I download and unpack the appropriate file from the Nutch website
( http://nutch.apache.org/downloads.html ):
[hadoop@hc1r1m2 Downloads]$ ls -l apache-nutch-2.2.1-src.tar.gz
-rw-rw-r--. 1 hadoop hadoop 3839858 Apr 7 18:32 apache-nutch-2.2.1-src.tar.gz
 
Search WWH ::




Custom Search