Database Reference
In-Depth Information
Listing 6-7. The show tables command
hive> show tables;
OK
aaplstockdata
hivesampletable
stock_analysis
stock_analysis1
Time taken: 3.182 seconds, Fetched: 4 row(s)
You can create new tables, populate them based on the data files in your blob containers in different partitions
and query them based on different criteria directly from the Hive console. However, using .NET SDK and PowerShell
are the recommended ways of making Hive job submissions in HDInsight rather than running them interactively from
the console.
Note
Details of Hive operations are covered in Chapter 8 of this topic.
The Sqoop Console
Sqoop is an Open Source Apache project that facilitates bi-directional data exchange between Hadoop and any
traditional Relational Database Management System (RDBMS) . It uses the MapReduce framework under the hood
to perform import/export operations, and often it is a common choice for integrating data from relational and
nonrelational data stores. In this section, we take a quick look at Sqoop operations that are compatible with Microsoft
SQL Server on Azure.
Sqoop is based on Java Database Connectivity (JDBC) technology to establish connections to remote RDBMS
servers. Therefore, you need the JDBC driver for SQL Server to be installed.
Table 6-1 summarizes a few of the key Sqoop operations that are supported with SQL Server databases in Azure.
Table 6-1. Sqoop commands
Command
Function
The import command lets you import SQL Server data into WABS. You can opt to import an
entire table using the --table switch or selected records based on criteria using the --query
switch. The data, once imported to the Azure storage system, is stored as delimited text files or
as SequenceFiles for further processing. You can also use the import command to move SQL
Server data into Hive tables, which are like logical schemas on top of WASB.
sqoop import
You can use the export command to move data from WASB into SQL Server tables. Much like
the import command, the export command lets you export data from delimited text files,
SequenceFiles, and Hive tables into SQL Server. The export command supports inserting new
rows into the target SQL Server table, updating existing rows based on an update key column, as
well as invoking a stored procedure execution.
sqoop export
The job command enables you to save your import/export commands as a job for future
re-use. The saved jobs remember the parameters that are specified during execution, and they
are particularly useful when there is a need to run an import or export command repeatedly on
a periodic basis.
sqoop job
To quickly check the version of sqoop you are on, you can run the sqoop version command to
print the installed version details on the console.
sqoop version
 
Search WWH ::




Custom Search