Database Reference
In-Depth Information
SnapshotTime :
Context : Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageContext
Name : example/data/WordCountOutputPS/part-r-00000
human 57
humana, 1
humane, 1
humani 2
humanist). 1
humanists 1
humanorum 1
inhuman 1
l'humano 1
Depending on your computer's security policies, you may get an exception, as shown next, when you try to run
PowerShell scripts that use
.dll
files that are compiled and signed externally:
PS C:\> .\SubmitJob.ps1
.\SubmitJob.ps1 : File C:\SubmitJob.ps1 cannot be loaded because running scripts is disabled on this
system. For more
information, see about_Execution_Policies at
http://go.microsoft.com/fwlink/?LinkID=135170
.
At line:1 char:1
+ .\SubmitJob.ps1
If you encounter such a problem, you need to explicitly set the PowerShell execution policy using the following
command:
Set-ExecutionPolicy RemoteSigned
While setting the execution policy, accept any warnings you might get in the PowerShell console. It is also
possible to submit Hive jobs using PowerShell, much like the .NET SDK. Carl Nolan has a great blog that covers Hive
job submission through PowerShell:
Using MRRunner
To submit MapReduce jobs, HDInsight distribution offers a command-line utility called
MRRunner
, which could be
utilized as well apart from the .NET SDK and the HDInsight PowerShell cmdlets. Again, to support the MRRunner
utility, you should have an assembly (a .NET .dll) that defines at least one implementation of
HadoopJob<>
.
If the .dll contains only one implementation of
HadoopJob<>
, (like our
HadoopClient.dll
does), you can run the
job with the following:
MRRunner -dll MyDll
If the .dll contains multiple implementations of
HadoopJob<>
, you need to indicate the one you wish to run:
MRRunner -dll MyDll -class MyClass
