Database Reference
In-Depth Information
14/10/28 21:36:28 INFO mapreduce.Job: Running job:
job_1413746845532_0008
14/10/28 21:36:35 INFO mapreduce.Job: Job job_1413746845532_0008
running in
uber mode : false
14/10/28 21:36:35 INFO mapreduce.Job: map 0% reduce 0%
14/10/28 21:36:41 INFO mapreduce.Job: map 100% reduce 0%
14/10/28 21:36:41 INFO mapreduce.Job: Job job_1413746845532_0008
completed
successfully
...
14/10/28 21:36:41 INFO mapreduce.ImportJobBase: Retrieved 3 records.
Sqoop's
import
tool will run a MapReduce job that connects to the MySQL database
and reads the table. By default, this will use four map tasks in parallel to speed up the im-
port process. Each task will write its imported results to a different file, but all in a com-
mon directory. Because we knew that we had only three rows to import in this example,
we specified that Sqoop should use a single map task (
-m 1
) so we get a single file in
HDFS.
We can inspect this file's contents like so:
%
hadoop fs -cat widgets/part-m-00000
1,sprocket,0.25,2010-02-10,1,Connects two gizmos
2,gizmo,4.00,2009-11-30,4,null
3,gadget,99.99,1983-08-13,13,Our flagship product
NOTE
The connect string (
jdbc:mysql://localhost/hadoopguide
) shown in the example will read
from a database on the local machine. If a distributed Hadoop cluster is being used,
localhost
should
not be specified in the connect string, because map tasks not running on the same machine as the data-
base will fail to connect. Even if Sqoop is run from the same host as the database sever, the full host-
name should be specified.
By default, Sqoop will generate comma-delimited text files for our imported data. Delim-
iters can be specified explicitly, as well as field enclosing and escape characters, to allow
the presence of delimiters in the field contents. The command-line arguments that specify
delimiter characters, file formats, compression, and more fine-grained control of the im-
in the online help (
sqoop help import
, or
man sqoop-import
in CDH).
