Databases Reference
In-Depth Information
The Generic JDBC Connector does not use catalog tables and views, and so it doesn't
have issues with synonyms. Instead it will issue a query with the clause
WHERE 1=0
that
won't transfer any data as the condition is always false but will return correct metadata
for transported data. Returned metadata will contain the basic information required,
like column count, names, and associated types; however, it lacks any advanced infor‐
mation like whether the table is partitioned or not. Although the Generic JDBC Con‐
nector works quite nicely here, it can't take full advantage of your database server.
7.9. Faster Transfers with Oracle
Problem
Sqoop does a great job transferring data between Oracle and Hadoop. Is there a faster
and more optimal way of exchanging data with Oracle?
Solution
You should consider using OraOop, a specialized connector for Oracle developed and
maintained by Quest Software, now a division of Dell. You can download the connector
from the
Cloudera website
.
Discussion
OraOop is a highly specialized connector for the Oracle database. Instead of splitting
data into equal ranges using one column (usually the table's primary key), OraOop
utilizes the concept of
rowid
. In doing so, the connector ensures that no two parallel
running tasks will read data from the same Oracle block. This lowers disk operations
on the database server, significantly improving performance. You are encouraged to
download, install, and use the OraOop connector instead of the built-in one.
See Also
Detailed instructions about the installation of special connectors are covered in
Recipe 1.3
.
7.10. Importing into Avro with OraOop
Problem
You are importing a table containing a
DATE
column from Oracle database into Avro
format, but you're getting the following exception:
org.apache.avro.UnresolvedUnionException: Not in union ["long","null"]:
