Databases Reference
In-Depth Information
-rw-rw-r-- 1 hadoop-user hadoop 2065 Dec 1 10:07 capacity-scheduler.xml
-rw-rw-r-- 1 hadoop-user hadoop 535 Dec 1 10:07 configuration.xsl
-rw-rw-r-- 1 hadoop-user hadoop 49456 Dec 1 10:07 hadoop-default.xml
-rwxrwxr-x 1 hadoop-user hadoop 2314 Jan 8 17:01 hadoop-env.sh
-rw-rw-r-- 1 hadoop-user hadoop 2234 Jan 2 15:29 hadoop-site.xml
-rw-rw-r-- 1 hadoop-user hadoop 2815 Dec 1 10:07 log4j.properties
-rw-rw-r-- 1 hadoop-user hadoop 28 Jan 2 15:29 masters
-rw-rw-r-- 1 hadoop-user hadoop 84 Jan 2 15:29 slaves
-rw-rw-r-- 1 hadoop-user hadoop 401 Dec 1 10:07 sslinfo.xml.example
The first thing you need to do is to specify the location of Java on all the nodes includ-
ing the master. In hadoop-env.sh
environment variable to point
to the Java installation directory. On our servers, we've it defined as
define the
JAVA_HOME
export JAVA_HOME=/usr/share/jdk
(If you followed the examples in
chapter 1,
you've already completed this step.)
The hadoop-env.sh file contains other variables for defining your Hadoop
environment, but
JAVA_HOME
is the only one requiring initial modification. The default
settings on the other variables will probably work fine. As you become more familiar
with Hadoop you can later modify this file to suit your individual needs (logging
directory location, Java class path, and so on).
The majority of Hadoop settings are contained in XML configuration files. Before
version 0.20, these XML files are hadoop-default.xml
and hadoop-site.xml. As the
names imply, hadoop-default.xml contains the default Hadoop
settings to be used
unless they are explicitly overridden in hadoop-site.xml. In practice you only deal with
hadoop-site.xml. In version 0.20 this file has been separated out into three XML files:
core-site.xml, hdfs-site.xml, and mapred-site.xml. This refactoring better aligns the
configuration settings to the subsystem of Hadoop that they control. In the rest of this
chapter we'll generally point out which of the three files used to adjust a configuration
setting. If you use an earlier version of Hadoop, keep in mind that all such configuration
settings are modified in hadoop-site.xml.
In the following subsections we'll provide further details about the different
operational modes of Hadoop and example configuration files for each.
2.3.1
Local (standalone) mode
The standalone mode
is the default mode for Hadoop. When you first uncompress the
Hadoop source package, it's ignorant of your hardware setup. Hadoop chooses to be
conservative and assumes a minimal configuration. All three XML files (or hadoop-
site.xml before version 0.20) are empty under this default mode:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
</configuration>
