Database Reference
In-Depth Information
Log in to the cluster. Any of the 11 servers, including CM, is good for that, because
the Gateway service is installed on each one of them. In my case, the login command
looks as follows:
mark@mark-dev:~$ ssh -i .ssh/shmsoft-hadoop.pem ubuntu@ec2-50-17-135-
161.compute-1.amazonaws.com
Once there, I can look around HDFS as follows:
ubuntu@ip-10-180-215-163:~$ hdfs dfs -ls /
Found 3 items
drwxr-xr-x - hbase hbase 0 2014-12-30 03:41 /hbase
drwxrwxrwt - hdfs supergroup 0 2014-12-30 03:45 /tmp
drwxr-xr-x - hdfs supergroup 0 2014-12-30 03:43 /user
However, if you try to create your home directory, it won't work:
hdfs dfs -mkdir /user/ubuntu
mkdir: Permission denied: user=ubuntu, access=WRITE,
inode=""/user"":hdfs:supergroup:drwxr-xr-x
To fix this, you need to do the following (as described at
https://github.
com/hadoop-illuminated/HI-labs/tree/master/hadoop-admin/managers/
cloudera-cm
):
ubuntu@ip-10-180-215-163:~$ sudo -u hdfs hdfs dfs -mkdir /user/ubuntu
ubuntu@ip-10-180-215-163:~$ sudo -u hdfs hdfs dfs -chown ubuntu /user/
ubuntu
ubuntu@ip-10-180-215-163:~$ hdfs dfs -mkdir /user/ubuntu/mark
Now you have your home, and in fact, your user directory (
mark
in my case, so that I
can see it):
ubuntu@ip-10-180-215-163:~$ hdfs dfs -ls
Found 1 items
drwxr-xr-x - ubuntu supergroup 0 2014-12-30 04:03 mark
Moreover, I can even put files there. For example, I can put my install file in
mark/
,
as follows:
hdfs dfs -put cloudera-manager-installer.bin mark/
And, lo and behold, I can see that file:
hdfs dfs -ls mark
Found 1 items
