Submitting Jobs to Your HDInsight Cluster - Pro Microsoft HDInsight: Hadoop on Windows - page 73

■ Note You may need to comment out a few of the other function calls to avoid repetitive operations.

Execute the code and you should see output similar to Listing 5-11.

Listing 5-11. Hive job output

Executing Hive Job.........Done..List of Tables are:
aaplstockdata
hivesampletable
stock_analysis
stock_analysis1

■ the hivesampletable is the only table that comes built in as a sample. i have other tables created, so your
output may be different based on the hive tables you have.

Note

The .NET APIs provide the .NET developers the flexibility to use their existing skills to automate job submissions
in Hadoop. This simple console application can be further enhanced to create a Windows Form application and
provide a really robust monitoring and job submission interface for your HDInsight clusters.

Monitoring Job Status
The .NET SDK also supports the Hadoop supporting package Ambari . Ambari is a framework that provides
monitoring and instrumentation options for your cluster. To implement the Ambari APIs, you need to add the NuGet
package Microsoft.Hadoop.WebClient . You will also need to import the following namespaces in
your Program.cs file:

using Microsoft.Hadoop.WebClient.AmbariClient;
using Microsoft.Hadoop.WebClient.AmbariClient.Contracts;

Once the references are added, create a new function called MonitorCluster() and add the code snippet as
shown in Listing 5-12.

Listing 5-12. MonitorCluster() method

public static void MonitorCluster()
{
var client = new AmbariClient(Constants.azureClusterUri,
Constants.clusterUser, Constants.clusterPassword);
IList<ClusterInfo> clusterInfos = client.GetClusters();
ClusterInfo clusterInfo = clusterInfos[0];
Console.WriteLine("Cluster Href: {0}", clusterInfo.Href);

Regex clusterNameRegEx = new Regex(@"(\w+)\.*");
var clusterName = clusterNameRegEx.Match(Constants.azureClusterUri.Authority).Groups[1].
Value;
HostComponentMetric hostComponentMetric = client.GetHostComponentMetric(
clusterName + ".azurehdinsight.net");