Database Reference
In-Depth Information
Then, to enable passwordless login, generate a new SSH key with an empty passphrase:
%
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
% cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
You may also need to run
ssh-add
if you are running
ssh-agent
.
Test that you can connect with:
%
ssh localhost
If successful, you should not have to type in a password.
Formatting the HDFS filesystem
Before HDFS can be used for the first time, the filesystem must be formatted. This is done
by running the following command:
%
hdfs namenode -format
Starting and stopping the daemons
To start the HDFS, YARN, and MapReduce daemons, type:
%
start-dfs.sh
%
start-yarn.sh
%
mr-jobhistory-daemon.sh start historyserver
NOTE
If you have placed configuration files outside the default
conf
directory, either export the
HADOOP_CONF_DIR
environment variable before running the scripts, or start the daemons with the
--
config
option, which takes an absolute path to the configuration directory:
%
start-dfs.sh --config
path-to-config-directory
%
start-yarn.sh --config
path-to-config-directory
%
mr-jobhistory-daemon.sh --config
path-to-config-directory
start historyserver
The following daemons will be started on your local machine: a namenode, a secondary
namenode, a datanode (HDFS), a resource manager, a node manager (YARN), and a his-
tory server (MapReduce). You can check whether the daemons started successfully by
looking at the logfiles in the
logs
directory (in the Hadoop installation directory) or by
looking at the web UIs, at http://localhost:50070/ for the namenode, http://localhost:8088/
