Database Reference
In-Depth Information
The TaskTracker logs come into play when the Hive queries are through the physical plan-generation phase and
into the MapReduce phase. From that point forward, TaskTracker logs will have a detailed tracing of the operations
performed. Note that the individual tasks are executed on the data nodes, hence the TaskTracker logs are available in
the data nodes only.
The NameNode maintains the log files for the JobTracker service in the same
C:\apps\dist\hadoop-1.2.0.1.3.0.1-0302\logs\
. folder. The JobTracker service is responsible for determining
the location of the data blocks, maintaining co-ordination with and monitoring the TaskTracker services running on
different data nodes. The file name is
Hadoop-jobtracker-<node name>.log
. You can open the file, and its contents
should be similar to Listing 13-12.
Listing 13-12.
The JobTracker Log
2013-11-16 17:28:29,781 INFO org.apache.hadoop.mapred.JobTracker:
Initializing job_201311120315_0003
2013-11-16 17:28:29,781 INFO org.apache.hadoop.mapred.JobInProgress:
Initializing job_201311120315_0003
2013-11-16 17:28:29,952 INFO org.apache.hadoop.mapred.JobInProgress:
Input size for job job_201311120315_0003 = 5015508. Number of splits = 1
2013-11-16 17:28:29,953 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201311120315_0003_m_000000 has split on node:/fd0/ud0/localhost
2013-11-16 17:28:29,953 INFO org.apache.hadoop.mapred.JobInProgress:
job_201311120315_0003 LOCALITY_WAIT_FACTOR=0.25
2013-11-16 17:28:29,953 INFO org.apache.hadoop.mapred.JobInProgress:
Job job_201311120315_0003 initialized successfully with 1 map tasks and 1 reduce tasks.
2013-11-16 17:28:29,966 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (JOB_SETUP) 'attempt_201311120315_0003_m_000002_0' to tip
task_201311120315_0003_m_000002, for tracker 'tracker_workernode2:127.0.0.1/127.0.0.1:49175'
2013-11-16 17:28:37,865 INFO org.apache.hadoop.mapred.JobInProgress:
Task 'attempt_201311120315_0003_m_000002_0' has completed task_201311120315_0003_m_000002 successfully.
2013-11-16 17:28:37,869 INFO org.apache.hadoop.mapred.JobInProgress:
Choosing a non-local task task_201311120315_0003_m_000000
2013-11-16 17:28:37,870 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
'attempt_201311120315_0003_m_000000_0' to tip task_201311120315_0003_m_000000,
for tracker 'tracker_workernode2:127.0.0.1/127.0.0.1:49175'
2013-11-16 17:28:39,710 INFO org.apache.hadoop.mapred.JobInitializationPoller:
Removing scheduled jobs from waiting queuejob_201311120315_0003
2013-11-16 17:28:42,118 INFO org.apache.hadoop.mapred.JobInProgress:
Task 'attempt_201311120315_0003_m_000000_0' has completed task_201311120315_0003_m_000000 successfully.
2013-11-16 17:28:42,151 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (REDUCE) 'attempt_201311120315_0003_r_000000_0' to tip
task_201311120315_0003_r_000000, for tracker 'tracker_workernode2:127.0.0.1/127.0.0.1:49175'
2013-11-16 17:28:51,473 INFO org.apache.hadoop.mapred.JobInProgress:
Task 'attempt_201311120315_0003_r_000000_0' has completed task_201311120315_0003_r_000000 successfully.
2013-11-16 17:28:51,484 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (JOB_CLEANUP) 'attempt_201311120315_0003_m_000001_0' to tip
task_201311120315_0003_m_000001, for tracker 'tracker_workernode2:127.0.0.1/127.0.0.1:49175'
2013-11-16 17:28:53,734 INFO org.apache.hadoop.mapred.JobInProgress:
Task 'attempt_201311120315_0003_m_000001_0' has completed task_201311120315_0003_m_000001 successfully.
2013-11-16 17:28:53,735 INFO org.apache.hadoop.mapred.JobInProgress:
Job job_201311120315_0003 has completed successfully.