Database Reference
In-Depth Information
As with the Statestore, the Catalog Service is not mission critical. If the
Catalog Service is down for any reason, users would need to execute
REFRESH
table after performing an insert or
INVALIDATE METADATA
after
DDL operations on any Impala daemon they were connecting to.
Is Impala Open Source?
The simple answer is yes. Impala is an open source product. However,
there is a catch. It's an extension to CDH (Cloudera Distribution
Including Apache Hadoop). This last point is important. You cannot
use Impala on any old Hadoop distribution; it is unique to Cloudera. So
although it is open source, it is in many ways proprietary. Just because
something is open source doesn't mean that there is no vendor lock-in.
Like Hortonworks, Cloudera monetizes their investment in Hadoop
through support and training. Impala is no exception. Real Time Query
(RTQ) is the technical support package for Impala and is an extension
of Cloudera Enterprise (their base enterprise technical support
offering). To get RTQ, you have to purchase both Cloudera Enterprise
and RTQ.
Microsoft's Contribution to SQL in Hadoop
Microsoft's initial contribution was really focused on getting Hadoop
(specifically Hortonworks HDP) running on Windows. This has become
the basis upon which HDInsight, Microsoft's platform as a service Hadoop
offering, has been built. More recently, Microsoft has been collaborating
with Hortonworks on the Stinger initiative. Personally speaking, that is
clear given the obvious commonality between SQL Server Column Store
and batch mode processing and Hadoop's OCFile optimizations and the
vectorized query processing seen in Tez. Tez also introduces a more general,
expressive, cost-based optimizer for executing complex directed acyclic
graphs (DAGs). Because SQL Server has one of the most sophisticated and
complex cost-based optimizers on the market today, I am certain that the
team will be able to make a significant and positive contribution to this new
processing paradigm for Hadoop.