Database Reference
In-Depth Information
In the above example, the HDFS file /test/test_file.txt has been copied to the Linux file system as ./test_file2.txt.
To move a file or directory on HDFS, you use the mv command:
[hadoop@hc1nn ~]$ hadoop fs -mv /test/test_file.txt /test/test_file3.txt
[hadoop@hc1nn ~]$ hadoop fs -ls /test
Found 1 items
-rw-r--r-- 2 hadoop hdfs 504 2014-03-24 18:24 /test/test_file3.txt
The HDFS file /test/test_file.txt has been moved to the HDFS file /test/test_file3.txt. You can recursively delete in
HDFS by using rm -r:
[hadoop@hc1nn ~]$ hadoop fs -rm -r /test
[hadoop@hc1nn ~]$ hadoop fs -ls /
Found 4 items
drwxrwxrwt - hdfs hadoop 0 2014-03-23 14:58 /tmp
drwxr-xr-x - hdfs hadoop 0 2014-03-23 16:06 /user
drwxr-xr-x - hdfs hadoop 0 2014-03-23 14:56 /var
The example above has deleted the HDFS directory /test and all of its contents. To determine the space usage in
HDFS, you use the du (disk usage) command:
[hadoop@hc1nn ~]$ hadoop fs -du -h /
0 /tmp
4.5 M /user
0 /var
The -h option just makes the numbers humanly readable. This last example shows that only the HDFS file system
/user directory is using any space.
Hadoop User Commands
This section introduces some Hadoop user commands that you can use to check the health of the HDFS, determine
the Hadoop version, and carry out large-scale distributed data copies. Hadoop fsck offers the ability to determine
whether the file system is healthy. It displays the total data size plus file and directory volumes. It also offers
information like the replication factor and corrupted blocks. Hadoop distcp provides the functionality to move very
large volumes of data between clusters.
The following example of the fsck command shows that the file system “/” is healthy. No corrupted or under-
replicated blocks are listed. By default, there should be two copies of each block saved (the default replication factor
value was 2). If the HDFS had failed in this area, it would be shown in the report as “Under-replicated blocks” with a
value greater than zero.
 
Search WWH ::




Custom Search