Database Reference
In-Depth Information
do a good job in helping you effectively and efficiently get your data into the
cloud.
Integrating Cloud Data
Onceyourdataisstoredinthecloud,integrating itintoyourHadoopcluster
is a trivial task. Vendors have a done a lot of work to ensure that the
links between their cloud storage and cloud computing services are not only
seamless but also high performance.
BothS3andAzureBlobdatacanbereferencednativelywithineachvendor's
respective service using a path that starts with either
s3://
or
asv://
as
the path prefix.
Within Microsoft HDInsight, there are three use cases to consider when
integrating cloud data stored in Azure Blob Storage, as described next.
Local Container and Containers on the Same Storage Account
Data can be created and read from both the same container and other
containers on the same storage account that is used to run your HDInsight
cluster. In this scenario, the key required to read from the storage account
is already stored in the
core-site.xml
. This allows for full access to the
data within these containers.
Public Container
When a container (and the blobs it contains) is marked as public, you can
access data directly by specifying a URI path to the blob. The path required
varies slightly from the one introduced previously. When accessing a file
from a URI, the path format looks like the following:
http://<Account>.blob.core.windows.net/<Container>/
<Path>
TheURIrequiredforaccesstothesamefilefromHDInsightwouldtranslate
to the following:
asv://<Container>@<Account>.blob.core.windows.net/
<Path>
