Database Reference
In-Depth Information
Chapter 12
Troubleshooting Cluster Deployments
Once you really start to play around with your HDInsight clusters, you are bound to end up with problems. Whether
the problems are related to the manual or programmatic deployments of clusters or submitting your MapReduce jobs,
troubleshooting is the art of logically removing the roadblocks that stand between you and your Big Data solution.
This chapter will focus specifically on common cluster-deployment failure scenarios and ways to investigate them.
Cluster Creation
As you saw in Chapter 3, creating a cluster using either
Quick Create
or
Custom Create
involves a sequence of
operations that need to be successfully completed to make the cluster operational. The phases are marked by the
status shown at each stage:
•
Submitting
•
Accepted
•
Windows Azure VM Configuration
•
HDInsight Configuration
•
Running
Table
12-1
explains what goes on behind the scenes during each of these phases.
Table 12-1.
Status designations when creating HDInsight clusters
Status
What it means
Submitting
The communication in this step is between the Azure portal and the HDInsight
Deployment Service, which is a REST API provided by HDInsight in Azure for its internal
use in these kind of operations. If there is a failure here, it is likely a problem with the
parameters of the setup, or a serious failure of the Deployment service.
Accepted
The HDInsight Deployment Service orchestrates the actions from this point forward,
communicating status back to the Azure portal. A hidden Cloud Service is provisioned
as a container, and then Cluster Storage is set up using your Storage Account and Key.
A container is then created with a default name matching the storage account name.
(Note: You can customize Storage Account links if you wish.) When this is successful, all
the preconditions for setup have been met. If you encounter a failure at this step, it is highly
likely that you have provided incorrect storage account details or duplicate container name.
(
contuinued
)
