Collecting Data with Nutch and Solr - Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset - page 63

export CLASSPATH=.:$NUTCH_HOME/runtime/local/lib
export CLASSPATH=$CLASSPATH:$NUTCH_HOME/conf
export PATH=$PATH:$NUTCH_HOME/bin
You can now build Nutch using Ant. Move to the Nutch installation directory (/usr/local/nutch) using the
NUTCH_HOME variable you just set up. Issue the ant command to start the build. The build output will be copied to the
session window:
[hadoop@hc1nn nutch]# cd $NUTCH_HOME
[hadoop@hc1nn nutch]# ant
Buildfile: build.xml
.....
BUILD SUCCESSFUL
Total time: 11 minutes 15 seconds
It can take quite a while to run the build, depending on the volume of changes and the age of the build; for
instance, this example took me more than 11 minutes. Checking the job file in the Nutch build directory shows (by
date and size) when the Nutch job file has been re-created:
[hadoop@hc1nn home]$ cd $NUTCH_HOME/build
[hadoop@hc1nn build]$ ls -l *.job
-rw-r--r--. 1 hadoop hadoop 79104396 Apr 1 18:34 apache-nutch-1.8.job
With Nutch installed and ready, you can move on to installing Solr, in preparation for using these tools together.
Solr Installation
To begin your installation, download Solr from the Solr website ( https://lucene.apache.org/solr ). For example,
I selected the zipped file to download and easily unpacked it using the Linux unzip command:
[root@hc1nn Downloads]# ls -l solr-4.7.0.zip
-rw-rw-r--. 1 hadoop hadoop 157644303 Mar 29 13:08 solr-4.7.0.zip
The release unpacks to a directory called solr-4.7.0 in the same location as the .zip file.
[root@hc1nn Downloads]# unzip solr-4.7.0.zip
[root@hc1nn Downloads]# ls -ld solr-4.7.0
drwxr-xr-x. 7 root root 4096 Feb 22 08:39 solr-4.7.0
Move the release to /usr/local/ (the same location as your Nutch and Hadoop software) and change its
ownership and group membership to hadoop. (Remember to use the recursive -R flag with the Linux chown
command.) As with the other software releases, a symbolic link called “solr” is created to point to the installed release.
[root@hc1nn Downloads]# mv solr-4.7.0 /usr/local
[root@hc1nn Downloads]# cd /usr/local
[root@hc1nn local]# chown -R hadoop:hadoop solr-4.7.0
[root@hc1nn local]# ln -s solr-4.7.0 solr