Database Reference
In-Depth Information
Job Counters
Launched map tasks=1
Other local map tasks=1
Total time spent by all maps in occupied slots (ms)=17353
Total time spent by all reduces in occupied slots (ms)=0
Map-Reduce Framework
Map input records=20031
Map output records=20031
Input split bytes=87
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=97
CPU time spent (ms)=3230
Physical memory (bytes) snapshot=103755776
Virtual memory (bytes) snapshot=823398400
Total committed heap usage (bytes)=43712512
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=1427076
14/07/21 15:53:52 INFO mapreduce.ImportJobBase: Transferred 1.361 MB in 39.9986 seconds
(34.8419 KB/sec)
14/07/21 15:53:52 INFO mapreduce.ImportJobBase: Retrieved 20031 records.
14/07/21 15:53:52 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `rawdata`
AS t LIMIT 1
14/07/21 15:53:52 INFO hive.HiveImport: Loading uploaded data into Hive
14/07/21 15:53:56 INFO hive.HiveImport: Logging initialized using configuration in file:/etc/hive/
conf.dist/hive-log4j.properties
14/07/21 15:53:56 INFO hive.HiveImport: Hive history file=/tmp/hadoop/hive_job_log_ca16a3e3-8ece-
4b54-8c5b-e4da6779e121_58562623.txt
14/07/21 15:53:57 INFO hive.HiveImport: OK
14/07/21 15:53:57 INFO hive.HiveImport: Time taken: 0.958 seconds
14/07/21 15:53:59 INFO hive.HiveImport: Loading data to table default.rawdata
14/07/21 15:54:00 INFO hive.HiveImport: Table default.rawdata stats: [num_partitions: 0,
num_files: 2, num_rows: 0, total_size: 1427076, raw_data_size: 0]
14/07/21 15:54:00 INFO hive.HiveImport: OK
14/07/21 15:54:00 INFO hive.HiveImport: Time taken: 2.565 seconds
14/07/21 15:54:00 INFO hive.HiveImport: Hive import complete.
14/07/21 15:54:00 INFO hive.HiveImport: Export directory is empty, removing it.
As you can see, the data was imported to HDFS and then successfully imported to Hive. The warnings in
this output indicate that you could have used a --direct flag in the import that would employ MySQL-specific
functionality for a faster import. (I didn't use the flag so as to demonstrate a simple data import; I wasn't worried
about performance.) To check the table's contents, use the following commands:
hive>
> show tables;
OK
rawdata
Time taken: 1.344 seconds
Search WWH ::




Custom Search