Database Reference
In-Depth Information
Chapter 7
Using Windows Azure HDInsight
Emulator
Deploying your Hadoop clusters on Azure invariably incurs some cost. The actual cost of deploying a solution
depends on the size of your cluster, the data you play with, and certain other aspects, but there are some bare-minimum
expenses for even setting up a test deployment for evaluation. For example, you will at least need to pay for your Azure
subscription in order to try the HDInsight service on Azure. This is not acceptable for many individuals or institutions
who want to evaluate the technology and then decide on an actual implementation. Also, you need to have a test bed to
test your solutions before deploying them to an actual production environment. To address these scenarios, Microsoft
offers the Windows Azure HDInsight Emulator.
The Windows Azure HDInsight Emulator is an implementation of HDInsight on the Windows Server family.
The emulator is currently available as a Developer Preview, where the Hadoop-based services on Windows use
only a single-node deployment. HDInsight Emulator provides you with a local development environment for the
Windows Azure HDInsight Service. It uses the same software bits as the Azure HDInsight service and is the test bed
recommended by Microsoft for testing and evaluation.
â–  While it's technically possible create a multinode configuration of HDInsight emulator, doing so is neither a
recommended nor a supported scenario, because it opens the door to serious security breaches in your environment.
If you are still eager to do the multinode configuration and you delete the firewall rule and modify the *-conf.xml Hadoop
config files, you'll essentially be allowing anyone to run code on your machine and access your file system. However,
such a configuration can be tested in a less sensitive lab environment solely for testing purposes and is documented in
the following blog post: http://binyoga.blogspot.in/2013/07/virtual-lab-multi-node-hadoop-cluster.html .
Caution
Like the Azure service, the emulator is also based on Hortonworks Data Platform (HDP), which bundles all the
Apache projects under the hood and makes it compatible with Windows. This local development environment for
HDInsight simplifies the configuration, execution, and processing of Hadoop jobs by providing a PowerShell library
with HDInsight cmdlets for managing the cluster and the jobs run on it. It also provides a .NET SDK for HDInsight for
automating these procedures—again, much like the Azure service. For users who need multinode Hadoop solutions
on their on-premises Windows servers today, the recommended option is to use HDP for Windows. Microsoft has no
plans whatsoever to make this emulator multinode and give it the shape of a production on-premises Hadoop cluster
on Windows.
 
 
Search WWH ::




Custom Search