Database Reference
In-Depth Information
■
Do not forget the supporting mapreduce classes
SquareRootMapper
,
SquareRootReducer
,
SquareRootJob
,
and
Constants
.
Note
Using PowerShell
Apart from the .NET Framework, HDInsight also supports PowerShell cmdlets for job submissions. As of this writing,
the Azure HDInsight cmdlets are available as a separate download from the Microsoft download center. In the future,
it will be a part of
Windows Azure PowerShell version 0.7.2
and there will be no separate download.
Windows Azure
HDInsight PowerShell
can be downloaded from:
Writing Script
For better code management and readability, let's define a few PowerShell variables to store the path of the
.dll
files
you will refer to throughout the script:
$subscription = "Your_Subscription_Name"
$cluster = "democluster"
$storageAccountName = "democluster"
$Container = "democlustercontainer"
$storageAccountKey = Get-AzureStorageKey $storageAccountName | %{ $_.Primary }
$storageContext = New-AzureStorageContext -StorageAccountName $storageAccountName
-StorageAccountKey $storageAccountKey
$inputPath = "wasb:///example/data/gutenberg/davinci.txt"
$outputPath = "wasb:///example/data/WordCountOutputPS"
$jarFile = "wasb:///example/jars/hadoop-examples.jar"
$class = "wordcount"
$secpasswd = ConvertTo-SecureString "Your_Password" -AsPlainText -Force
$myCreds = New-Object System.Management.Automation.PSCredential ("admin", $secpasswd)
The sequence of operations needed to move you toward a job submission through PowerShell is pretty much the
same as in the .NET client:
Creating the job definition
•
•
Submitting the job
•
Waiting for the job to complete
•
The following piece of PowerShell script does that in sequence:
Reading and displaying the output
# Define the word count MapReduce job
$mapReduceJobDefinition = New-AzureHDInsightMapReduceJobDefinition -JarFile $jarFile -ClassName
$class -Arguments $inputPath, $outputPath
# Submit the MapReduce job
Select-AzureSubscription $subscription
