Database Reference
In-Depth Information
#Wait for the job to complete
$pigJob | Wait-AzureHDInsightJob -Subscription $subid -WaitTimeoutInSeconds 3600
Using the Grunt shell in Pig is another way to bypass coding MapReduce jobs, which can be tedious and time
consuming. The HDInsight name node gives you the option to interactively run Pig commands from their respective
command shells. Doing so is often a great way to isolate any external issues when you are facing errors while
submitting jobs using PowerShell or .NET.
Hadoop Web Interfaces
Core Hadoop provides a couple of web interfaces to monitor your cluster and, by default, they are available at the
desktop of the name node. These portals can provide useful details about the cluster health, usage and MapReduce
job execution statistics. The shortcuts to these portals are created on the desktop during the Azure virtual machine
(VM) provisioning process, as shown in Figure 6-10 . They are
Hadoop MapReduce Status
Hadoop Name Node Status
Figure 6-10. Shortcuts to the web portals
Hadoop MapReduce Status
The Hadoop MapReduce portal displays information on job configuration parameters and execution statistics in
terms of running/completed/failed jobs. The portal also shows job history log files. You can drill down on individual
jobs and examine the details.
The portal is referred to as the JobTracker portal, because each MapReduce operation is submitted and executed as
a job in the cluster. The tracker portion of the portal is basically a Java-based web application that listens on port 50030.
The URL for the portal is http://<NameNode_IP_Address>:50030/jobtracker.jsp . Figure 6-11 shows the
MapReduce status or the JobTracker status portal when it is launched.
 
Search WWH ::




Custom Search