Database Reference
In-Depth Information
blocks about data files in HDFS. Each Impala node caches all of the metadata loc-
ally, which can expedite the process of gathering metadata for a large amount of
data, distributed across multiple DataNodes. When dealing with an extremely large
amount of data and/or many partitions, getting table specific metadata could take
a significant amount of time. So a locally stored metadata cache helps in providing
such information instantly.
When a table definition or table data is updated, other Impala daemons must update
their metadata cache by retrieving the latest metadata before issuing a new query
against the table in question. Impala uses
REFRESH
when new data files are added
to an existing table. Another statement,
INVALIDATE METADATA
, is also used when
a new table is included, or an existing table is dropped. The same
INVALIDATE
METADATA
statement is also used when data files are removed from HDFS or a DFS
rebalanced operation is initiated to balance data blocks in HDFS.
The Impala programming interface
Impala provides the following ways to submit queries to the Impala daemon:
• Command-line interface through Impala shell
• Web interface through Apache Hue
• Third-party application interface through ODBC/JDBC
The Impala daemon process is configured to listen to incoming requests from the
previously described interfaces via several ports. Both the command-line interface
and web-based interface share the same port; however, JDBC and ODBC use dif-
ferent ports to listen for the incoming requests. The use of ODBC- and JDBC-based
connectivity adds extensibility to Impala running on the Linux environment. Using
ODBC and JDBC third-party applications running on Windows or other Linux plat-
forms can submit queries directly to Impala. Most of the third-party Business Intelli-
gence applications use JDBC and ODBC to submit queries to the Impala cluster and
the
impalad
processes running on various nodes listen to these requests and pro-
cess them as requested.
