Database Reference
In-Depth Information
The Cat Command
The Hadoop file system
cat
command copies the contents of the URIs presented to it to STDOUT.
hdfs dfs -cat hdfs://hc1nn/user/hadoop/oozie_wf/fuel/pigwf/manufacturer.pig
The
cat
command is useful if you want to run adhoc Linux-based commands against Hadoop-based data.
For instance, the following:
hdfs dfs -cat hdfs://hc1nn/user/hadoop/oozie_wf/fuel/pigwf/manufacturer.pig | wc -l
would give you a line count of this file using the Linux command
wc
(word count ), with a
-l
switch for the number
of lines.
The CopyFromLocal Command
The source for the
CopyFromLocal
command is the local Linux file system; this command copies files from the local
Linux file system to Hadoop.
[hadoop@hc1nn flume]$ hdfs dfs -copyFromLocal /home/hadoop/flume /tmp/flume
[hadoop@hc1nn flume]$ hdfs dfs -ls /tmp/flume
Found 6 items
-rw-r--r-- 2 hadoop hadoop 1343 2014-07-26 20:09 /tmp/flume/agent1.cfg
-rw-r--r-- 2 hadoop hadoop 1483 2014-07-26 20:09 /tmp/flume/agent1.cfg.nl
-rw-r--r-- 2 hadoop hadoop 45 2014-07-26 20:09 /tmp/flume/flume_clean_hdfs.sh
-rw-r--r-- 2 hadoop hadoop 197 2014-07-26 20:09 /tmp/flume/flume_exec.sh
-rw-r--r-- 2 hadoop hadoop 233 2014-07-26 20:09 /tmp/flume/flume_exec.sh.nl
-rw-r--r-- 2 hadoop hadoop 42 2014-07-26 20:09 /tmp/flume/flume_show_hdfs.sh
In this example, the contents of the Linux file system directory /home/hadoop/flume have been copied to HDFS
under /tmp/flume.
The CopyToLocal Command
For the
CopyToLocal
command, the destination directory must be the local Linux file system. This command copies
files from HDFS to the Linux file system:
[hadoop@hc1nn flume]$ hdfs dfs -copyToLocal /tmp/flume /tmp/hdfscopy
[hadoop@hc1nn flume]$ ls -l /tmp/hdfscopy
total 24
-rwxr-xr-x. 1 hadoop hadoop 1343 Jul 26 20:13 agent1.cfg
-rwxr-xr-x. 1 hadoop hadoop 1483 Jul 26 20:13 agent1.cfg.nl
-rwxr-xr-x. 1 hadoop hadoop 45 Jul 26 20:13 flume_clean_hdfs.sh
-rwxr-xr-x. 1 hadoop hadoop 197 Jul 26 20:13 flume_exec.sh
-rwxr-xr-x. 1 hadoop hadoop 233 Jul 26 20:13 flume_exec.sh.nl
-rwxr-xr-x. 1 hadoop hadoop 42 Jul 26 20:13 flume_show_hdfs.sh
The contents of the /tmp/hdfscopy directory on HDFS have been copied to the Linux file system directory /tmp/
hdfscopy.
Search WWH ::
Custom Search